Feasibility and acceptability of the brief patient-reported experience measure consideRATE within the hospital setting for patients with palliative care needs,their families/carers and clinicians

Abstract

Background:

Supporting clinical teams to improve care for inpatients with palliative care needs and their carers is a known priority. Patient reported experience measures (PREMs) may assist in improvement work. Evidence about how to collect and feedback PREM data for this population and context is required.

Aim:

To determine the feasibility of implementing a brief, validated PREM, consideRATE and appraise its acceptability as perceived by inpatients with palliative care needs, their carers and clinicians.

Design:

A prospective study using: 1) PREM administration, screening log and field note completion; and; 2) a focus group with clinicians.

Setting/participants:

Eligible participants recruited from three wards (cancer care and internal medicine) of an Australian tertiary metropolitan hospital. Participants included patients screened to have palliative care needs (using the SPICT^TM criteria), their carers and multidisciplinary clinicians (including clinical managers).

Results:

Feasibility: A 71% response rate was achieved (n = 80 from 112 eligible patients approached). Mean screening time to inform eligible patients for PREM completion was 7.5 min. More than half of eligible participants (n = 47, 59%) opted for electronic completion of consideRATE and mean completion time was 6.12 min. A third of participants required assistance for PREM completion (n = 27, 34%). Score distribution varied across response options, albeit with a positive skew towards ‘very good’ and ‘good’. Two thirds of respondents (n = 50, 62.5%) provided ⩾1 free-text response. Acceptability: Clinicians valued consideRATE data noting feedback needed to be: accessible, supported by free-text and responsive to local contexts.

Conclusions:

It is feasible to implement consideRATE for inpatients with palliative care needs. Clinicians note consideRATE data is acceptable in informing improvement foci.

Keywords

Palliative care hospital patient reported experience measures patient-centred care quality improvement quality of care

What is already known about the topic?

In high-income countries approximately 30% of inpatients have palliative care needs and require improved access to integrated palliative care.

Patient reported experience measures (PREMs) provide a useful tool for informing improvement efforts but implementation of these within the context of hospital based palliative care is untested.

What this paper adds?

It is feasible to administer a PREM (consideRATE – designed for patients with palliative care needs) within the acute care setting for patients with palliative care needs (71% response rate achieved).

Screening for patient eligibility using the SPICT^TM criteria is time efficient (7.5 min) and identifies the right population of patients to approach for PREM completion.

Data to note a large number of patients with palliative care needs (34%) require assistance for PREM completion and a variance of format was preferred (59% preferring electronic format and 41% paper).

Implications for practice, theory or policy?

Recommendations for future practice

Amplifying the patient voice through the collection of PREM data (using consideRATE) from inpatients with palliative care needs is feasible and ought to guide practice improvement work.

When designing how to integrate consideRATE into improvement work, who administers the PREM, in what format and how the data is fed back to clinical teams needs careful consideration.

Introduction

Palliative care provision for the 30%^1
–4 of inpatients noted to be in their last 12 months of life remains a priority area for improvement globally.^5,6 Routine use of patient-reported data may assist in improvement of care quality.^3
–5,7 Patient Reported Experience Measures (PREMs) capture patients’ perspectives about their care.⁸ In the palliative context, ensuring PREMs align with patient and carer priorities is crucial^9
–11 to meaningful participation and data.

Work led by several members of this research team has revealed that inpatients with palliative care needs and their carers require the following key domains to be present to enable high-quality and safe palliative care (Textbox 1).^11
–14 Patients with palliative care needs are defined as having ⩾2 general indicators of poor or deteriorating health and ⩾1 clinical indicator of one or multiple life-limiting conditions as listed within the Supportive and Palliative Care Indicators Tool (SPICT^TM).¹⁵

Textbox 1.

Domains of importance for high-quality and safe palliative care.

1. Respectful and compassionate care

2. Effective communication and shared decision making

3. Effective teamwork

4. Enabling family involvement

5. Maintaining role, meaning and identity

6. Excellence in physical care

7. Impeccable assessment and care planning

8. Effective symptom management

9. Technical competence

10. Patient safety

11. Supported access to senior clinicians

12. Patient focused structural factors (e.g., restful area, access to a window, consider implications of shared rooms)

13. Family focused structural factors (e.g., access to seating, a place to lie at night and simple kitchen items)

14. Cleanliness to support infection control

Alignment between these domains and PREMs is vital if improvement work is to remain focused on what matters most for people with palliative care needs. Our recent systematic review exploring PREMs designed for inpatients with palliative care needs noted: 44 PREMs available addressing between 1 and 11 of the 14 domains of importance.¹⁰ In addition to ensuring alignment with what matters most for this population, accessibility across the broader population is also of paramount importance. Analysis of PREMs designed for inpatients with palliative care needs found the majority of items (62%) are written at grades higher than those recommended for the broader population.¹⁰ People with palliative care needs experience significant cognitive load due to illness and treatments, so may struggle to comprehend PREMs with high readability scores.^10,16 The need for a tailored PREM for use outside of specialist palliative care is well stated.^7,10 However, to the best of our knowledge limited evidence exists relating to the implementation of PREMs within research or clinical settings in the context of palliative care. A recent scoping review confirms this with their finding of no PREMs cited within work focused on assessing the needs for people with severe frailty nearing the end of life.⁷

Our recent study explored the perspectives of inpatients with palliative care needs, their carers and the clinical team regarding the use of a generic PREM compared to one tailored to what matters most for people with palliative care needs. We found that a tailored, brief and easy-to-use PREM with free-text space was preferred for people with palliative care needs.⁹ One of the previously identified PREMs¹⁰ met these criteria, the eight item consideRATE¹⁷ measure, noting this focuses on patient reported experience (carer measurement in proxy) and not on carer needs. consideRATE¹⁷ was published in 2021 after development by a team in the United States across three phases: 1. establishing what is most important to patients; 2. Refining items via cognitive interviewing; 3. Piloting the measure with patients and families. Following this development work reliability and validity of this measure was tested via an online simulation study¹⁸ and recent additional validation work has determined convergent validity when compared with another validated PREM (CANHELP-Lite).¹⁹ Exploring the feasibility and acceptability for consideRATE within a real-world context was not yet established and informed the importance of this study.

This study aimed to determine the feasibility of implementing a brief, validated patient-reported experience measure, consideRATE, within the Australian hospital inpatient setting, and appraise its acceptability as perceived by inpatients with palliative care needs, their carers and clinicians.

Method

Data reported in this manuscript comes from multiphase research exploring the feasibility and acceptability of a brief experience measure within an Australian hospital for patients with palliative care needs, their carers and clinicians. Detailed methods and results have been published in relation to the earlier phase of this study (exploring the optimal tool for use and agreeing to test consideRATE).⁹ This manuscript reports the feasibility and acceptability of implementing consideRATE.

Research questions

What are the screening and administration requirements for implementing consideRATE for inpatients with palliative care needs?

What do clinical teams require in relation to feedback of data captured from consideRATE to guide clinical service development?

Design

A prospective acceptability and feasibility study²⁰ Study design included: 1) PREM administration, screening log and field note completion; and; 2) a focus group with clinicians.

Setting

Eligible participants were recruited from three wards in two departments (cancer care and internal medicine) of an Australian tertiary metropolitan hospital.

Population

PREM administration: Inpatients and family members were eligible if: an adult (aged ⩾ 18) with palliative care needs within one of the participating hospital wards, or an adult carer; and able to provide informed consent.

Focus group: Clinicians were eligible if they were: 1) clinicians (nurses, doctors or allied health professionals) from participating wards; or 2) hospital executives or quality management personnel from participating departments (either cancer care or internal medicine).

Sample

Convenience sampling was used. All patients within each participating ward were screened by a research nurse against the eligibility criteria (Table 1). Once screening was completed, the research nurse checked screening outcomes with the nursing unit manager (or their delegate) to confirm eligibility and check in relation to potential cognitive impairments. All clinicians and managers working within the participating wards were eligible for inclusion in the study.

Table 1.

Inclusion and exclusion criteria for patient and carers.

Inclusion	Exclusion
Adult (aged >18 years)	Cognitive impairment meaning the patient could not consent to participation and no carer present
Screened to be ‘SPICT’ positive and therefore have palliative care needs. ‘SPICT positive’ was defined as ⩾2 general indicators of poor or deteriorating health and ⩾1 clinical indicator of one or multiple life-limiting conditions	Completed the PREM within the past week
Inpatient within one of the three participating wards for >24 h	Patient or family distressed and the nurse leader felt it was inappropriate to approach them at this time
Carer of a patient screened to be eligible but cognitively unable to complete the PREM
Eligible patient or carer who has completed the PREM previously but more than a week ago

Recruitment

Inpatients and carers: Eligible patients were approached after screening and invited to participate. Data was collected by a research nurse who was available for 2 days each week. She would approach eligible patients within her data collection times and if patients were not on the ward or asleep at time of collection, they would not be approached.

Clinicians: An initial email (and two follow-up reminders) with key information about the study was sent to eligible stakeholders by the lead researcher, inviting participation.

No incentive was offered for participation in this study.

Patient Reported Experience Measure (PREM)

consideRATE¹⁷ is a brief, validated PREM designed to measure perceived quality of care for inpatients with serious illness.¹⁸ consideRATE has been validated in English within the US context only at this point with this work completed in an online simulation format.¹⁸ Informed by our earlier systematic review^13,17 and codesigned with patients and families, consideRATE¹⁷ maps to domains of care that matter most for people with palliative care needs. consideRATE’s eight questions are listed in Textbox 2.

Textbox 2.

ConsideRATE’s^17,18 eight questions.

Q1 How would you rate our attention to your physical problems? things like pain, dry mouth or trouble breathing

Q2 How would you rate our attention to your feelings? things like feeling sad, worried or like a burden

Q3 How would you rate our attention to your surroundings? things like noise, light or warmth

Q4 How would you rate our respect for what matters to you? things like values, preferences about care or important activities

Q5 How would you rate our communication about your plans? things like medicines, procedures or place of care

Q6 How would you rate our attention to your affairs? things like financial planning and preparing enduring documents

Q7 How would you rate our attention to what you can expect? things like illness getting worse or time left to live

Q8 Are there any other things you want to share?

The instructions provided for patients and carers refer to this PREM being for ‘people who are ill or their caregivers’ and that it is referring to ‘the care you had from our hospital in the last few weeks’.¹⁷ Response options included ‘very good’, ‘good’, ‘bad’, ‘very bad’ and ‘doesn’t apply’. A modified version of consideRATE¹⁷ was used with free-text response options added per question. This modification was completed by this research team with approval from the original authors of consideRATE. No significant changes to consideRATE question wording, question prompts and measurement scale were made.

Data collection

consideRATE¹⁷ was administered to eligible and available inpatients throughout November and December 2022 in either paper or electronic format (given a tablet preloaded with consideRATE within Qualtrics). Patients were offered support via the research nurse to complete consideRATE (they could either self-complete or the research nurse could read this to them and note their responses). If a patient could not complete due to cognitive impairment, their carer was eligible to complete on their behalf. The completion of consideRATE was anonymous and participants were informed of this prior to completion. Demographic data collected included: age, gender, SPICT positive criteria and diagnosis. Feasibility data (research question 1) focused on screening and survey completion: 1) screening – time taken to screen ward; number of eligible patients for tool completion; and number of eligible patients and carers who participated; 2) survey completion – format preference from the patient or carer perspective (paper or electronic), time taken to complete the survey and whether assistance was required.

The focus group informed research question 2, was facilitated by one member of the research team, audio-recorded, transcribed verbatim and field notes taken to assist in understanding aspects of participant interactions. The interview guide focused on views about: usefulness of data; useful presentation options to guide improvement work; and implementation barriers or facilitators (Textbox 3). Demographic data were collected to describe study sample, including: age, gender, professional role and highest level of education.

Textbox 3.

Focus group interview question route.

1. What are your views about the reporting available after piloting the implementation of a brief patient experience tool?2. Which report format is most usable for you?3. Does this tool provide useful data for you to work with?4. Are there any parts of this tool that concern you or that you would like changed?5. How do you think this tool could be administered within the clinical environment?6. What are the key enablers and barriers we need to consider if we want to implement this tool across the hospital?7. Any other comments or questions about the tool itself and reporting options?8. Any other comments about what you feel would help clinical teams focus on improving inpatient palliative care across the hospital?

Data analysis

PREM data: Descriptive statistics were used to explore completion rates, time taken, support required to complete, respondent characteristics and distribution of responses as key indices of feasibility. Free-text data were analysed independently by two researchers (CV and ID) to understand: which questions generated more free-text responses; whether responses provided constructive feedback to inform change and, if so, what key area for improvement was noted; whether the rating made was reflected in the free-text feedback (i.e., if a participant noted a ‘good’ rating but provided critical feedback); and whether comments provided were in alignment with the question rated. Consensus meetings with the full research team were held to finalise results and discuss areas where the independent free text analysis was not in alignment.

Clinician focus group: Sekhon et al.’s²¹ theoretical framework of acceptability informed deductive coding.²² The focus group transcript was checked against the audio-file, before coding within Excel line-by-line. Coding and theme development was completed by one reviewer (CV) and checked by a second reviewer (TL) to achieve consensus. Following this, review by additional members of the research team provided feedback, further developed themes and resolved areas requiring consensus.

Ethics

Ethical approval: July 2021: HREC/2021/QRBW/77494.

Results

Screening and recruitment

Of 717 patients screened, 38% (n = 270) met the criteria for palliative care needs (SPICT positive). After review of these patients with the nurse in charge, 249 were deemed eligible (n = 21 excluded due to cognitive impairment with no carer present or distress). 112 patients were approached (research nurse approached as many eligible and available patients as possible across 2 days per week) and 80 participated (71% participation rate) – Figure 1. Participant demographics are provided in Table 2. Given this study is exploring the use of consideRATE for patients in non-specialist palliative care units (general wards), a process to identify who can complete the PREM is needed and this study used screening via the SPICT^TM tool. The mean time taken to screen each inpatient from a ward handover sheet was 7.5 min (range 2–12 min). The mean time to discuss screening results and gain consensus on eligible patients for study participation with the nurse in charge was 2.3 min (range 1–4 min).

Figure 1.

Screening and recruitment flow diagram.

Table 2.

Demographic details for patients (n = 80) noting 5 patients, 6%, had carers report in proxy.

Gender	Age	Diagnoses
42 males (52.5%), 38 females (47.5%)	29–92 years (mean age 65, SD 12)	• Malignancy (n = 48; 60%)• Renal failure (n = 17; 21%)• Multi-morbidity (malignancy and non-malignancy) (n = 5, 6%)• Multi-morbidity (all non-malignancy) (n = 4; 5%)• Heart failure (n = 2; 3%)• Parkinson’s disease (n = 1; 1.25%)• Dementia (n = 1; 1.25%)• Liver failure (n = 1; 1.25%)• Chronic obstructive pulmonary disease (n = 1; 1.25%)

PREM administration and completion

Given a choice of paper or electronic surveys, more than half of the participants (n = 47, 59%) opted for electronic. Mean time taken to complete consideRATE was 6.12 min (range 44 s to 17.49 min, median = 5.14 min). Although most participants did not require assistance to complete the survey (n = 53; 66%), one-third did. The reasons participants sought assistance either related to physical limitations (n = 24; 30%) or literacy issues (n = 3; 4%).

Distribution of scores varied across response options (Figure 2), albeit with a positive skew towards ‘very good’ and ‘good’. Many ‘doesn’t apply’ ratings were made for the questions relating to: attention to affairs (30/80, 38%); and attention to what the person might expect in relation to illness getting worse or time left to live (17/80, 21%). Only 4/80 (5%) had missing data, in response to questions two, three, four and seven.

Figure 2.

ConsideRATE responses.

Most participants (n = 50, 62.5%) provided at least one free-text response. The questions participants added most comments to related to environment (n = 26, 32%), attention to what they can expect (n = 19, 24%) or feelings (n = 19, 24%). Most free-text comments aligned with the question asked (⩾94%). Disparity between the rating and related comment was noted in some instances, with evidence of patients rating ‘good’ but providing a constructive comment alongside this. This disparity occurred for 50% of comments made about environmental aspects and 41% of comments made about communication. These results are summarised in Table 3 and Supplemental Table S1.

Table 3.

Analysis of rating and free-text data.

consideRATE question	Number of free-text comments added (%)	Number of free-text comments that could inform change/improvement (%)	Number of comments that aligned with noted rating (%)	Number of comments that aligned directly with question focus (%)
Q1 – Attention to symptoms	15/80 (19)	5/15 (33)	10/15 (67)	15/15(100)
Q2 – Attention to feelings	19/80 (24)	4/19 (21)	15/19 (79)	18/19 (95)
Q3 – Environmental aspects	26/80 (32)	20/26 (77)	13/26 (50)	26/26 (100)
Q4 – Attention to respect	9/80 (11)	2/9 (22)	8/9 (89)	9/9 (100)
Q5 – Communication	17/80 (21)	12/17 (70)	10/17 (59)	16/17 (94)
Q6 – Advance care planning	18/80 (22)	14/18 (78)	16/18 (89)	18/18 (100)
Q7 – Attention to what to expect	19/80 (24)	11/19 (58)	15/19 (79)	19/19 (100)

Denominators refer to number of free-text comments made from the full participant sample (e.g. of the 80 PREMs completed, 15 participants provided free text comments about question 1).

Final free-text question: Are there any other things you want to share?

Almost half of participants, 49% (39/80), chose to provide a free-text comment about any other comments they wanted to share. Of those, 38% (15/39) made comments that could inform change, and remaining comments were positive (Supplemental Table S2). Six areas were identified for improvement:

care integration

physical care

communication

meaningful engagement for patients to distract from boredom and worries

nutrition to enhance wellbeing and recovery

timely person-centred, holistic and respectful care

Focus group results – Clinician perspectives on feedback requirements and usefulness of data captured from consideRATE

Eight clinicians contributed to the focus group (Table 4).

Table 4.

Demographic details of focus group participants (n = 8).

Professional discipline	Age	Gender	Educational level
• 6 nurses (Clinical Nurse Consultant in Palliative care, Clinical Nurse Consultant in Safety and Implementation, Nursing Unit Manager, Clinical Nurse, Registered Nurse × 2)• 2 medical practitioners (Medical Oncologist and Palliative Medicine Consultant).	53 years (range from 33 to 69 years)	Female (n = 6, 75%), male (n = 2, 25%)	Bachelors degree (n = 4, 50%) or post-graduate qualifications (n = 4, 50%)

Four themes were identified: 1) PREM data feedback is valued provided it is in an accessible, brief format and supported by free-text; 2) Processes for PREM data collection and feedback need to be responsive to clinician and patient needs; 3) Clinicians value PREM data that is specific enough to both context and patient population to inform improvement work; and 4) Clinicians valued consideRATE whilst noting it could be improved with changes to the rating scale, role for carer responses and some wording.

1) PREM data feedback is valued provided it is in an accessible, brief format and supported by free-text

Clinicians value data that is easy to read and can be viewed rapidly to provide a snapshot of current quality of care:

For me it’s easy enough to get a bit of an understanding of what’s going on by looking at it in this format. So I might need it, some more information if I was trying to look at it all the time from a more analytical point of view. But it gives me a pretty good snapshot of what’s going on and I can see whether things are good or bad or . . . indifferent. (44 year female, Clinical Nurse Consultant)

They found the addition of free-text data invaluable in providing deeper insights into ratings and defining key areas for action noting quantitative ratings alone did not always enable this:

I think per question is nice because then you’ve got a particular topic area to work on. Like when I’m looking at some of the comments here were providing the comfort around noise, light, everything else. . . I think having specific comments like that is also valuable for us to take back to the [team] who are trying to be mindful of minimising noise and even around the sort of inconveniences to patients trying to get some sleep that would be valuable to us and I like that a lot. (56 year female, Clinical Nurse Consultant)

When reviewing free-text data they appreciated simple categorising of the data into positive and constructive comments, for easier scanning:

I did wonder if you had the comments presented just in a table with positive, constructive or something like that as the two, it would help you scan that perhaps more easily. (Facilitator initiated with a chorus of agreement across all attendees, n = 8)

2) Processes for PREM data collection and feedback need to be responsive to clinician and patient needs

Clinicians noted their need for data feedback to be tailored for use within existing meeting structures:

No well it just depends really. I mean whether this is done to an inservice type and it might be one of the portfolio groups . . . or whether it’s a portion of it is presented say at our unit meetings. So if I had to take a portion of this that I would if I had my specific ward details I’d look at the noise, the noise level. And especially in that evening phase. And I would go to the specific comments about our ward and that’s what I would take. (56 year female, Clinical Nurse Consultant)

When to administer the PREM was noted to be important with clinicians feeling optimal patient engagement will occur when they are not acutely unwell:

. . . you wouldn’t want to be giving people a survey like this if they’re very unwell. So I wouldn’t see a lot of people would be able to engage with this when they’re having their acute treatments that have just come in and they’re sick. That’s not a good time, they need to be, as you were saying, coming closer to starting to talk about moving forward and going home or being you know transferred. (57 year Male, Medical Clinician)

Ensuring processes are in place to support patients to feel comfortable in providing honest feedback is a priority:

Can I go back a step and ask what is the question asked when you approach the bed and you have a patient and a carer there by the person is going to ask them to do the thing? Are you saying to the patient would you like to complete this yourself? Or are you saying would you like to work together to complete it? I think it’s going to get a different result. (57 year Male, Medical Clinician)

Importantly, clinicians noted the power imbalance between patients and clinical teams noting therefore that administration of the PREM ought to be by people not involved with current care provision:

You do then also have a bias of if they know they’re giving it straight back to the person who is going to read it is they tend you know they don’t want to give bad information to this wonderful person. (60 year Female, Nursing Unit Manager)

3) Clinicians value PREM data that is specific enough to both context and patient population to inform improvement work

When data is presented in an accessible way and is specific to identified patient populations, the clinical team find this valuable in informing improvement work:

I mean I guess it’s just a context of how you are interpreting results. Because . . . and it was more if that’s what it is designed for then that’s obviously the patients that you should use it in. I mean clearly the communication is another thing that’s very important for all the patients and the surroundings. But I guess if you’re doing it for patients without palliative care needs are there other things that they would see as important that aren’t in here that they want to tell us? (44 year female, Clinical Nurse Consultant)

Enabling specificity to the ward level opens opportunities for benchmarking and collaboration to support improved practice:

It is helpful I suppose to sort of do some bench marking with other areas to see how you’re going . . . it would be nice to see and if you know somebody else in another area doesn’t seem to be getting lots of negative comments about one particular thing it would be handy for me to pick up the phone or go over and see that NUM and say ‘What are you doing in this area’? (60 year Female, Nursing Unit Manager)

Finally, enabling access to trends in data is important for clinical teams to evaluate the impact of changes in practice:

My only question is about the ongoing. So if I’m wanting to look at trend over time. . . . So this is fantastic for responding to that particular period of survey. But if I want to know if I’m actually improving then I need to be able to see what happens over time. (44 year female, Clinical Nurse Consultant)

4) Clinicians valued consideRATE whilst noting it could be improved with changes to the rating scale, role for carer responses and some wording

Clinicians found the consideRATE survey to be acceptable and useful for their practice:

Oh I think it would just be very useful measure for us. So we could see, look at the areas where we don’t rate so well . . . Yeah so I would think it would be great to have this kind of data. (57 year Male, Medical Clinician)

Several comments were made in relation to the rating scale and the fact a neutral or average option was not available:

I think you’d nearly group them together and say anything good and very good is positive, at least it’s always hard to . . . there should be something between good and bad and all those sorts of things. (60 year Female, Nursing Unit Manager)

They also questioned the accuracy of carer’s completing this measure on behalf of patients:

. . . when I read this I’m not sure if they’d be answering this thinking . . . are they answering on behalf of the patient or are they answering on behalf of the carer. I think obviously it’s worded towards the patient. I’d be thinking if I was you know a carer, going am I . . . has my mum been given all the information or have I been given all the information . . . I’d be unsure how to answer it. when I read this I’m not sure if they’d be answering this thinking

Concern was noted about the questions 6 and 7 within consideRATE with clinicians concerned these questions may be difficult to administer and for patients to receive:

. . . yeah it’s not comfortable to have that conversation . . . Yeah and it’s very early and they might have just been diagnosed so that was a bit of a concern for us. (44 year female, Clinical Nurse Consultant)

Discussion

Main findings of the study

This study found using consideRATE¹⁷ with Australian inpatients with palliative care needs on cancer care and internal medicine wards was feasible. This study also provides exploratory work indicating it is acceptable from clinician perspectives for informing improvement work. This is important as to the best of our knowledge, this is the first study exploring the feasibility of collecting PREM data from inpatients with palliative care needs in general acute care ward contexts. consideRATE’s¹⁷ brevity and alignment with areas of importance for people with palliative care needs were central to its acceptability.⁹ The screening and data collection approach was found to be feasible, noting that this was conducted by funded nurse researchers rather than clinicians working on the ward with competing priorities. Identification of inpatients with palliative care needs was found to be feasible using the SPICT™²³ supported by a brief discussion with the nurse in charge.

What this study adds

Understanding how to best implement PREMs in various healthcare settings is evolving and yet to be fully realised.^8,24 Many studies have focused on collection of PREM data after an episode of care is completed²⁴ whereas this study provides unique understandings about real-time PREM completion from inpatients during an acute admission. Importantly, participants were assured anonymity both within the form itself (paper or electronic) but also through administration via a research nurse. Patients with palliative care needs and their carers have noted the importance of anonymity when appraising clinical teams.⁹ Acceptability of PREMs may be impacted if the data is not anonymous, such as if a treating team collects the data.

It is important to note that a significant number of patients requested assistance to complete consideRATE,¹⁷ mostly due to physical impairments. Electronic administration was hampered either by patient choice or poor internet coverage within the hospital. These factors need careful consideration when designing a PREM collection system as this study suggests electronic distribution at scale (e.g., texting a link to a survey) is unlikely to enable meaningful engagement for people with palliative care needs.

A distribution of ratings was obtained using consideRATE,¹⁷ albeit with a skew to positive ratings of ‘good’ or ‘very good’. However, there were a high number of patients rating question 6 (about advance care planning) or question 7 (about understanding what they might expect in relation to illness progression and expected prognosis) as not applying for them. Given these patients were identified to be living with an advanced progressive illness and likely to be in the last 12 months of their life, it would be valuable to understand why they did not see these questions as being relevant. It could be that these results reflect an interpretation issue as many patients noted enduring documents had been completed and so they may feel these need no further attention. The responses might also reflect inadequate communication from health professionals about a person’s overall health status and what they might expect, as reflected in broader literature.^11–14

The ability to add free-text responses to each question within consideRATE was informed by earlier patient and carer feedback noting this to be important.^9,16 This study confirmed this preference given that nearly two-thirds of patients provided comments. Comments made within the survey were sometimes repeated at the conclusion of the survey. This repetition might indicate that patients wanted to amplify this feedback. Additionally, ratings made within consideRATE did not always align with the nature of the free-text comment made. That is, many times a patient rated a question as ‘good’ but then provided a comment that did not reflect optimal care. This might be due to the lack of a ‘neutral’ or ‘average’ rating within the rating scale. Clinicians also noted the need for a ‘neutral’ or ‘average’ within the rating scale and felt this would provide greater discrimination between ratings to inform improvement. Whether to force choice or provide a mid-point option is hotly debated in survey development research.²⁵ Given the presence of a ‘doesn’t apply’ option, the consideRATE developers opted to use a forced-choice Likert-like response option set to avoid central tendency bias and social desirability bias.^26,27 This choice of response option set was theoretically driven, yet the optimal methods to reduce ceiling effects within PREMs remain unclear.²⁸

Using data from consideRATE to inform change was feasible from the perspectives of clinicians. As widely reported,^8,24 ensuring PREM data presentation was in a brief format and easily accessible for clinical teams was important. This study also confirmed the importance of PREM data that are both specific to patient population and clinical context to inform meaningful change. How to embed feedback into existing systems was noted to be both important and not clearly articulated. It is acknowledged that many PREMs exist to assist palliative care clinicians and services to appraise care quality.¹⁰ Furthermore, the scope and use of PREMs as opposed to other tools to identify need and appraise care delivery (such as patient reported outcome measures (PROMs) and needs assessment tools) was not explored. The need to better understand how PREMs and PROMs can be used together is noted as a current gap requiring further investigation.⁷ Collectively, providing explicit information about available tools, how they can be implemented and used to inform service improvement work for the acute care setting is important and urgent work for the large population of inpatients living with palliative care needs globally, and their carers.

Recommendations for future practice

Given mechanisms to drive sustained improvement in palliative care within the acute care setting continue to be elusive, understanding how to inform change founded in what matters most to patients with palliative care needs is vital. In response to learnings from this study, the following is recommended:

Implementation of a brief PREM tailored to the needs of patients with palliative care to inform focused improvement work. Who administers the PREM, in what format and how the data is fed back to clinical teams needs careful consideration.

Recommendations for future research

Validation work for consideRATE to explore consideration of a ‘neutral’ rating within the response options provided;

Exploratory work to better understand the high numbers of patients rating questions 6 and 7 as not applicable to their care;

Research to understand: the ideal data required to enable systematic improvements in clinical care delivery informed by patient and carer feedback; and what additional supports are needed for clinical teams to enact change in response to data provided (e.g., facilitation support).

Strengths and limitations of the study

This study has several strengths including the defined sampling and data analysis approach. Sampling across oncology and general medical wards enabled a broad sampling of people with palliative care needs in hospitals. Study limitations relate to the fact recruitment was limited by allocated research nurse time and so not all eligible patients were approached for survey completion. The research nurse worked the same 2 days each week across the study and this may have impacted patient recruitment although we note regular patient flow across all three wards independent of days of the week. In addition, screening for eligibility was informed by the SPICT²³ tool which may have led to inclusion of some patients who had a longer prognosis than 12 months. Views of people from culturally and linguistically diverse backgrounds, and Indigenous Australians were underrepresented. Focus group analysis was led by one author, supported by a deductive framework approach, with review from a second author and team consensus. This could have influenced interpretation. In addition, focus group data represented one group of clinicians and is therefore exploratory and warrants further investigation.

Conclusions

The use of PREMs to guide clinical improvement work is extensively documented as important in policy and literature^8,24,29 but specific guidance to support implementation is lacking. This study confirms that a brief PREM, consideRATE,¹⁷ is acceptable and feasible for use for inpatients with palliative care needs from the perspectives of patients and clinicians. Additional validation work of consideRATE¹⁷ is warranted, and cancer-specific validation results are forthcoming. In addition, further research is needed to inform how to use this PREM to inform clinical improvement work within busy ward environments.

Supplemental Material

sj-docx-1-pmj-10.1177_02692163241291343 – Supplemental material for Feasibility and acceptability of the brief patient-reported experience measure consideRATE within the hospital setting for patients with palliative care needs, their families/carers and clinicians

Supplemental material, sj-docx-1-pmj-10.1177_02692163241291343 for Feasibility and acceptability of the brief patient-reported experience measure consideRATE within the hospital setting for patients with palliative care needs, their families/carers and clinicians by Claudia Virdun, Elise Button, Jane L Phillips, Catherine H Saunders, Patsy Yates and Tim Luckett in Palliative Medicine

Supplemental Material

sj-docx-2-pmj-10.1177_02692163241291343 – Supplemental material for Feasibility and acceptability of the brief patient-reported experience measure consideRATE within the hospital setting for patients with palliative care needs, their families/carers and clinicians

Supplemental material, sj-docx-2-pmj-10.1177_02692163241291343 for Feasibility and acceptability of the brief patient-reported experience measure consideRATE within the hospital setting for patients with palliative care needs, their families/carers and clinicians by Claudia Virdun, Elise Button, Jane L Phillips, Catherine H Saunders, Patsy Yates and Tim Luckett in Palliative Medicine

Footnotes

Acknowledgements

The authors thank the patients and families who contributed time and such valuable information whilst living with or caring for someone living with palliative care needs. The authors also thank the three participating wards for their willingness for us to screen and recruit patients and families on their ward; and also for their participation in a focus group. We thank our research nurse (Rebecca Radford) for her work on administering consideRATE and collecting related feasibility data. We also thank Ingrid Duff (ID), a research assistant, for her support as a second coder within the data analysis. All authors contributed to the conceptualisation of this study and research protocol development (except CS). CV led the study conduct, supported the on-site research nurse, ran the focus group and led the data extraction and analysis. All authors participated in consensus discussions about data analysis and refinement of analytical themes (except CS). All authors contributed to the writing, editing and finalisation of this manuscript.

Declaration of conflicting interest

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Catherine H. Saunders reports copyright but no relevant financial interest in the consideRATE questions, the measure of serious illness care experience assessed in this study. All other authors declare that there is no conflict of interest.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Queensland University of Technology, Centre for Healthcare Transformation, 2021 Enabling Scheme Grant. This scholarship had no involvement in research design, conduct, manuscript preparation or choice of journal for submission.

Data management and sharing

Raw data are contained within this manuscript in the form of participant quotes and within supplemental files. The corresponding author is available to contact for further information. However, further data about each specific quote cannot be provided due to the risk of re-identifying the participants involved.

ORCID iDs

Claudia Virdun

Tim Luckett

Supplemental material

Supplemental material for this article is available online.

References

Mudge

Douglas

Sansome

, et al. Risk of 12-month mortality among hospital inpatients using the surprise question and SPICT criteria: a prospective study. BMJ Support Palliat Care 2018; 8: 213–220.

Greene

Agar

, et al. A point prevalence survey of hospital inpatients to define the proportion with palliation as the primary goal of care and the need for specialist palliative care. Intern Med J 2011; 41: 430–433.

Broad

Gott

Kim

, et al. Where do people die? An international comparison of the percentage of deaths occurring in hospital and residential aged care settings in 45 populations, using published and available statistics. Int J Public Health 2013; 58: 257–267.

Milnes

Orford

Berkeley

, et al. A prospective observational study of prevalence and outcomes of patients with Gold Standard Framework criteria in a tertiary regional Australian Hospital. BMJ Support Palliat Care 2019; 9: 92–99.

Australian Government Department of Health. National palliative care strategy 2018. Australia: Australian Government Department of Health, 2018. https://www.health.gov.au/sites/default/files/the-national-palliative-care-strategy-2018-national-palliative-care-strategy-2018.pdf

Connor S and Worldwide Hospice Palliative Care Alliance. Global atlas of palliative care. 2nd ed. London: Worldwide Hospice Palliative Care Alliance, 2020.

Howard

Green

Harris

, et al. Understanding the extent to which PROMs and PREMs used with older people with severe frailty capture their multidimensional needs: a scoping review. Palliat Med 2024; 38: 184–199.

Gilmore

Corazza

Coletta

, et al. The uses of patient reported experience measures in health systems: a systematic narrative review. Health Policy 2022; 128: 1–10.

Virdun

Button

Phillips

, et al. Perspectives of inpatients with palliative care needs, their families, clinicians and key stakeholders on measuring quality of hospital care via patient experience measures: a qualitative study. Palliat Med 2023; 37(10): 1498–1508.

10.

Virdun

Garcia

Phillips

, et al. Description of patient reported experience measures (PREMs) for hospitalised patients with palliative care needs and their families, and how these map to noted areas of importance for quality care: a systematic review. Palliat Med 2023; 37: 898–914.

11.

Virdun

Luckett

Davidson

, et al. Generating key practice points that enable optimal palliative care in acute hospitals: results from the OPAL project’s mid-point meta-inference. Int J Nurs Stud Adv 2021; 3: 100035.

12.

Virdun

Luckett

Davidson

, et al. Hospital patients’ perspectives on what is essential to enable optimal palliative care: a qualitative study. Palliat Med 2020; 34: 1402–1415.

13.

Virdun

Luckett

Davidson

, et al. Dying in the hospital setting: a systematic review of quantitative studies identifying the elements of end-of-life care that patients and their families rank as being most important. Palliat Med 2015; 29: 774–796.

14.

Virdun

Luckett

Lorenz

, et al. Dying in the hospital setting: a meta-synthesis identifying the elements of end-of-life care that patients and their families describe as being important. Palliat Med 2016; 31: 587–601.

15.

Highet

Crawford

Murray

, et al. Development and evaluation of the Supportive and Palliative Care Indicators Tool (SPICT): a mixed-methods study. BMJ Support Palliat Care 2014; 4: 285–290.

16.

Luckett

Virdun

Rao

, et al. Improving the methods for patient-reported experience measures in palliative care: findings from a cognitive interview study. Annals of Palliat Med 2022; 11: 2275–2284.

17.

Saunders

Durand

M-A

Scalia

, et al. User-centered design of the consideRATE Questions, a measure of people’s experiences when they are seriously ill. J Pain Symptom Manage 2021; 61: 555–565.e5.

18.

Saunders

Durand

M-A

Kirkland

, et al. Psychometric assessment of the consideRATE questions, a new measure of serious illness experience, with an online simulation study. Patient Educ Couns 2022; 105: 2581–2589.

19.

Heyland

Jiang

Day

, et al. The development and validation of a shorter version of the Canadian Health Care Evaluation Project Questionnaire (CANHELP Lite): a novel tool to measure patient and family satisfaction with end-of-life care. J Pain Sympt Manage 2013; 46: 289–297.

20.

Skivington

Matthews

Simpson

, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 2021; 374: 104705.

21.

Sekhon

Cartwright

Francis

JJ.

Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017; 17: 1–13.

22.

Azungah

Qualitative research: deductive and inductive approaches to data analysis. Qual Res J 2018; 18: 383–400.

23.

The University of Edinburgh. SPICTTM, https://www.spict.org.uk/ (2022, accessed 16 August 2018).

24.

Shunmuga Sundaram

Campbell

, et al. Patient and healthcare provider perceptions on using patient-reported experience measures (PREMs) in routine clinical care: a systematic review of qualitative studies. J Patient Rep Outcomes 2022; 6: 1–16.

25.

Chyung

Roberts

Swanson

, et al. Evidence-based survey design: the use of a midpoint on the Likert scale. Perform Improv 2017; 56: 15–23.

26.

Nadler

Weston

Voyles

EC.

Stuck in the middle: the use and interpretation of mid-points in items on questionnaires. J General Psychol 2015; 142: 71–89.

27.

Pornel

Saldaña

GA.

Four common misuses of the Likert scale. Philipp J Soc Sci Humanit 2013; 18: 12–19.

28.

Salman

Kopp

Thomas

, et al. What are the priming and ceiling effects of one experience measure on another? J Patient Exp 2020; 7: 1755–1759.

29.

Bull

Callander

EJ.

Current PROM and PREM use in health system performance measurement: still a way to go. Patient Exp J 2022; 9: 12–18.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.07 MB