Abstract
Background
The experiences and satisfaction of family members of patients are important indicators of healthcare quality in the intensive care unit. The family satisfaction in the intensive care unit (FS-ICU-24) questionnaire, developed in Canada and now validated in the UK, is becoming the gold standard measure to evaluate family members’ satisfaction with the intensive care unit. To inform future use of the UK FS-ICU-24 to evaluate quality improvement strategies aimed at improving family satisfaction with the intensive care unit, we set out to explore the extent to which the 24-scored items and domains of the UK FS-ICU-24 reflect common suggestions and priorities for quality improvement self-reported as important to family members in the UK.
Methods
Two data sources were thematically analysed – (1) open-text responses from family members who completed the UK FS-ICU-24 in a large observational cohort study; (2) a set of quality improvement activities generated by patients, family members and staff through experience-based co-design in a mixed-methods’ intensive care unit quality improvement study. Summarised themes were then mapped to the 24-scored items and domains of the UK FS-ICU-24 to assess coverage by the UK FS-ICU-24.
Results
We found a good degree of coverage between the topics and themes identified as important to family members with the 24-scored items and domains of the UK FS-ICU-24.
Conclusion
Our study confirms the face validity of the UK FS-ICU-24 and indicates that its inclusion as an outcome measure for evaluating quality improvement strategies aimed at improving family satisfaction with the intensive care unit is appropriate.
Introduction
Providing high-quality, person-centred care for the critically ill is challenging. Specifically, those aspects of quality, such as the humanity and equity of care, which involve communication and shared decision making may present challenges that are unique to critical care. Evaluating the experience of critical care is even more challenging – while some patients may be conscious during part of their stay and able to reflect on their own satisfaction with care, the majority are not. Furthermore, approximately one-fifth of patients do not survive their stay in the intensive care unit (ICU) and hospital. The relationship, therefore, between staff and family members (defined as persons who have a close familial, social or emotional relationship with the patient) is pivotal and, for these reasons, evaluating the experience of family members and their satisfaction with care for their loved one is an important measure of quality in the ICU.
The family satisfaction in the intensive care unit questionnaire (FS-ICU-24)1,2 is a self-completed, 24-scored item questionnaire used to measure family members’ satisfaction with the ICU. The FS-ICU-24 also includes three questions, at the end, inviting open-text responses. Originally developed with family member input in Canada, it is rapidly becoming the gold-standard. 1 The FS-ICU-24 was initially validated internationally3,4 and has now been validated in a large UK study.2,5 This resulted in a UK adaptation (UK FS-ICU-24) as psychometric assessment indicated three distinct domains (rather than two) – satisfaction with care, satisfaction with information and satisfaction with decision making. 6 While aimed at family members, many of the questions elicit information about the care of the patient. Improvements in family satisfaction are, therefore, likely to reflect in improvements in quality of care and, thus, family satisfaction could be an important outcome for evaluating quality improvement strategies in the ICU.
Although the UK FS-ICU-24 has been shown to detect important differences between ICUs in terms of family satisfaction, 2 as with all questionnaires, there may be issues around whether the items included fully capture the issues which matter most to family members about the quality of care of their loved one. In order to inform future use of the UK FS-ICU-24 to evaluate quality improvement strategies aimed at improving family satisfaction with the ICU, we set out to assess the extent to which the 24-scored items reflected common suggestions and priorities for quality improvement self-reported by family members in the UK. In order to do this, we compared the 24-scored items and domains of the UK FS-ICU-24 with themes identified from open-text comments collected as part of a large evaluation of family satisfaction with NHS critical care services, and with a set of quality improvement activities identified in a national ICU quality improvement study.
Methods
Data sources
Two family member-generated data sources, pertaining to quality improvement suggestions and priorities, were used. The first of these derived from the family reported experiences evaluation (FREE) study 2,5 and the second from a national experience-based co-design (EBCD) study.7,8
Briefly, FREE was a National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) Programme funded (reference: 11/2003/56) observational cohort study of family satisfaction with NHS critical care services in 20 representative ICUs. 2,5
Briefly, EBCD was an NIHR HS&DR Programme funded (reference: 10/1009/14) mixed-methods study in two ICUs involving co-design activities with patients (where able) and family members and staff to identify and implement priorities for quality improvement.7,8
FREE study
Raw UK FS-ICU-24 data were extracted for 1855 adult (aged 18 years or over) family members (self-identified as next of kin) of patients staying more than 48 h for the three questions inviting open-text response in the UK FS-ICU-24. These questions are
Do you have any suggestions on how to make care provided in the ICU better?
Do you have any comments on things we did well?
Please add any comments or suggestions that you feel may be helpful to the staff of this ICU.
Anticipating that the content of open-text responses would vary by overall family satisfaction score and to address representativeness,9,10 stratified sampling was used to ensure that responses from the full spectrum of overall level of satisfaction were included. Family members, overall satisfaction scores on the UK FS-ICU-24, were stratified into five groups, from which samples of open-text responses (of 50 characters or more) were drawn for thematic analysis.
EBCD study
Thirty-eight improvement activities were extracted from the report of the EBCD study.7,8 The improvement activities were synthesised through implementation of an accelerated version of EBCD. EBCD is a quality improvement approach which involves patients and family members in describing their experiences of care and then working in partnership with staff to design and implement service changes. This started with inductive analysis of national patient and family member recorded interviews to identity important ‘touchpoints’ or moments of interaction with the service where things could be improved which were then edited into a ‘trigger’ film. The trigger film was then shown at co-design workshops with former patients, family members and staff, from which top local improvement priority areas were identified. The improvement priorities were then taken forward by working groups of patients, family members and staff, which were formed to plan and implement specific improvements, resulting in the 38 improvement activities.
Thematic analysis
Based on a framework approach, first author SL (independent of both the FREE and EBCD studies), with supervision from co-author LL, undertook an inductive thematic analysis of the UK FS-ICU-24 open-text responses. 11 What is meant by inductive, in this case, is that analysis started from the family members’ own words in the open-text responses, looking for themes and patterns, rather than a deductive approach starting with the existing items and domains of the UK FS-ICU-24.
Open-text responses from each UK FS-ICU-24 respondent were read for immersion. An initial coding framework was developed to capture themes and developed further from reading the set of EBCD improvement activities (see above). 7 The framework consisted of overarching topics, each containing several themes. The data from both data sources were imported into the qualitative analysis software package QSR NVivo version 10 and coded systematically to the framework. The framework was refined and updated during the coding period. After coding was completed, the responses in each of the codes were summarised, with illustrations from the raw data.
Summarised themes, of the suggestions and priorities for quality improvement strategies aimed at improving family satisfaction with the ICU, were then mapped to the 24-scored items of the UK FS-ICU-24 (see Supplementary File – Table 1) to assess coverage by the UK FS-ICU-24 (face validity) by the first author (SL) and discussed with co-authors to reach consensus.
Results
Sampling for thematic analysis
UK FS-ICU-24 responses from the 1855 family members were stratified into five groups by overall satisfaction score (0–100) at score intervals (0 to <20, 20 to <40, 40 to <60, 60 to <80, 80 to 100; Figure 1). Most responses were in Group 5 (63%) with overall satisfaction scores of 80 and above. Less than 1% of responses were in Group 1 with overall satisfaction scores below 20. A purposive sample was then selected, including all family members in Groups 1 (n = 11) and 2 (n = 49) and, from each of Groups 3, 4 and 5, n = 60 randomly selected family members. Random selection used a pseudorandom number generator. The open-text responses included in the thematic analysis derived from the 240 family members.
Distribution of overall satisfaction score by Group for the 1855 family members.
Thematic analysis results
Coding framework –Topics and themes emerging from thematic analysis of two family-member generated data sources.
Illustrations of the inductively derived thematic analysis of the 240 open-text responses, in relation to selected topics and themes identified, are set out below.
The physical environment
Importance of rest and sleep
Rest and sleep were considered important for recovery. Family members were sometimes critical of noise levels in the unit and attempts to move patients during the night. Bright lights were also highlighted as a barrier, and eye covers suggested.
(FSSD15—160) My mother asked for an eye cover so she could sleep in the bright rooms she had to ask it wasn't offered maybe it should be part of your introduction to the patient to discuss this to make them as comfortable as possible, as quickly as possible in an [intimidating] atmosphere of medics to patients.
Care of patient and person
Knowing the person
Patients in the ICU are often unconscious or unable to communicate, making it particularly difficult for healthcare professionals to learn information about them: information that makes them a person, rather than just another patient.
12
Staff who knew the patient and communicated with them as such were appreciated by relatives.
(FSSD17—109) Some of the nurses kept calling my mother (the patient) 'Dear' as in “C'mon Dear” or “Take this dear”! I felt she would have responded better by being called her name.
(FSSD16—127) Certain nurses took the time to really get to know my mum and to find out about her - this really showed just how much they cared.
Care of family members
Improving contact with sick relatives
Access to their relative was often an issue. Doors to ICU are locked to protect vulnerable patients and their families, and staff members control entry. Some family members commented that they were left waiting, an inconvenience repeated over many visits. In addition, this was a waste of precious visiting hours, which family members already found restrictive. Long waits to see a family member were additionally distressing if no information was given as to the reason for the delay.
(FSSD15—63) Keep visitors better informed when delays in being able to see family member. I had to wait 3hrs on ICU—I was convinced he was dying.
(FSSD15—9) More involvement of close family— some family who care for their relatives when they were at home would like to be involved with the daily care whilst in ITU … Patients in ITU would often find it comforting to have a family member helping with their care and being with them more.
Orientation for family members
Family members valued regular progress updates from staff. Although family members had many complex information needs, such as whether or not their loved one would ever regain their former level of function, they also had simpler information needs such as where to find refreshments, how to get subsidies for parking, how to stay updated outside the hospital and how to help physically care for their relative. Though these may seem minor, they cause extra strain on family members already dealing with distress. Having limited or no experience with the hospital system, family members may not know or might be too shy to ask about amenities such as free parking and would prefer to have the information volunteered.
(FSSD17—85) We were not told anything about parking my mother was in ICU for 4 days and we weren’t told anything about free parking.
(FSSD17—88) Would have liked more information about my husband’s condition. This was offered when I asked but not volunteered very often.
Communication and decision making
Improving contact with doctors
A common complaint from family member respondents was that they found it difficult to contact doctors, and thus were excluded from the decision-making process and frustrated by a lack of information about their loved one.
(FSSD15—65) I was never briefed by a doctor. I had no time to ask questions about her care or progress from doctors. This continued till she was discharged. Nurses answered questions where they could. I never felt involved in her care …
(FSSD15—79) Communication with ITU doctors could be improved. Ward rounds and decisions are made in the absence of relatives. Either too early in the morning or too late after visiting time.
Care of staff
Staff support
Other than to comment about their communication skills and attentiveness, respondents did not discuss staff in many other ways. Some respondents did describe their appreciation of the extremely hardworking staff and suggested improving working conditions and paying them more. Staff being busy was recognised as a reason for lack of attention.
Comparison with UK FS-ICU-24 items
Topics, themes, results of UK FS-ICU-24 item-level comparisons and examples of quality improvement activities which may improve family satisfaction with the ICU.
Q denotes ‘question’ within the domain on the UK FS-ICU-24 (see Supplementary File – Table 1).
Derived from Locock et al. 6
We found a good degree of coverage between the themes identified as important to family members, from our thematic analysis, with the 24-scored items and domains of the UK FS-ICU-24 – both in terms of specificity and in the number of questions relating to the topics – particularly for the ‘care of patient and person’, ‘care of family members’ and ‘communication and decision-making’ topics.
Discussion
Principal finding
Our study found that the topics and themes, identified as important by family members of ICU patients in the UK, are represented by the 24-scored items and domains of the UK FS-ICU-24, confirming its face validity and indicating that its inclusion as an outcome measure for evaluating quality improvement strategies aimed at improving family satisfaction with the ICU is appropriate.
Strengths
A major strength of this study is the use of two data sources generated from individual family members with direct experience of ICU in the UK. It provides an inductively derived framework which can be used to underpin future quality improvement projects in the ICU, grounded in lived experience.
The topics and themes identified, through thematic analysis in our study, were similar to those identified in a qualitative study carried out in Canada, 13 suggesting that family members in the UK may have experiences that resonate in other healthcare systems. However, to our knowledge, ours is the only study to directly map resultant topics and themes onto the items and domains of the UK FS-ICU-24.
Limitations
The list of EBCD improvement activities, used as a data source in our study, may not reflect an exhaustive list of all potentially relevant quality improvement activities. However, the thematic analysis of the open-text responses from the FREE study highlighted similar topics and themes. Whether, or not, these topics and themes reflect the suggestions and priorities in future studies is difficult to determine given the inherent variability in co-designed quality improvement efforts. There will always be limits as to how far in advance we can anticipate exactly what needs to be measured. In some cases, the mapping of topics and themes onto the 24-scored items and domains of the UK FS-ICU-24 may be imperfect due to the subjective nature of some of the 24-scored items of the UK FS-ICU-24.
The UK adaptation of the FS-ICU-24, as a questionnaire to assess family satisfaction with the ICU, has three conceptual domains which assess satisfaction with care, information and with decision making – all within the ICU. As may be expected, no questions are included to directly evaluate the wider ‘hospital journey’ (e.g. family members’ experience of transfer or discharge, appropriate step-down care on other wards, etc.). As this topic relates to care received after the ICU, it is unlikely that quality improvement activities solely conducted within the ICU will change these family members’ experiences. This topic may also be influenced by quality improvement activities conducted elsewhere in the hospital, noting that family members have many experiences in other parts of the hospital or community that they relate and deem relevant to the ICU.
Conclusion
Based on qualitative analysis, our study found that the topics and themes identified as important by family members of ICU patients in the UK were represented by the UK FS-ICU-24 questionnaire. These results indicate that its inclusion as an outcome measure for evaluating quality improvement strategies aimed at improving family satisfaction with the ICU is appropriate.
Supplemental Material
INC883563 Supplemental Table1 - Supplemental material for Can the UK 24-item family satisfaction in the intensive care unit questionnaire be used to evaluate quality improvement strategies aimed at improving family satisfaction with the ICU? A qualitative study
Supplemental material, INC883563 Supplemental Table1 for Can the UK 24-item family satisfaction in the intensive care unit questionnaire be used to evaluate quality improvement strategies aimed at improving family satisfaction with the ICU? A qualitative study by Susannah Lyes, Alvin Richards-Belle, Bronwen Connolly, Kathryn M Rowan, Lisa Hinton and Louise Locock in Journal of the Intensive Care Society
Footnotes
Acknowledgements
We thank the participants in the FREE and EBCD studies who gave their time to participate. We also thank Professor David Harrison for extracting the open-text comments from the FREE study database.
Authors’ contributions
ARB, KMR and LL drafted the manuscript which was based on a report written by SL. SL conducted the qualitative analysis with supervision from LL. SL, ARB, BC, KMR, LH and LL read and approved the final manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The FREE Study was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) Programme (11/2003/56). The EBCD study was funded by the NIHR HS&DR Programme (10/1009/14). The funder had no involvement in study design; in collection, analysis and interpretation of data; in the writing of this paper; or in the decision to submit the article for publication. LH is supported by the NIHR Oxford Biomedical Research Centre (BRC). LL was employed by the Nuffield Department of Primary Care Health Sciences, University of Oxford and supported by the NIHR Oxford BRC during the period the work was conducted. BC is funded by an NIHR Clinical Trials Fellowship (CTF-2017-06-016). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR, NHS or the Department of Health and Social Care.
Research ethics and patient consent
Consent for analysis of the responses to the three questions inviting open-text response was already in place (National Research Ethics Service Committee South Central - Berkshire B (reference: 13/SC/0037)). The improvement priorities from the EBCD study are publicly available from the NIHR Journals Library report. Additional ethical approval was not required.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
