Abstract
Older people living with frailty emphasize autonomy and function as acute healthcare outcome goals. Existing Patient-Reported Outcome Measures (PROMs) measure function but do not comprehensively address autonomy. This initial development of a novel autonomy outcome measure used co-creation and cognitive interviews, working toward a PROM for Older People living with frailty receiving Acute Care (“PROM-OPAC”). Novel item question stems and responses considering autonomy were devised with lay research partners. Items were examined for content by lay volunteers, and then selected based on relevance, completeness, and accessibility. Retained items were cognitively tested with patient participants. Item selection considered content validity and feasibility and was undertaken collaboratively with lay research partners. The study involved 3 lay research partners and 4 further lay collaborators throughout all stages, and 14 patient participants were recruited for the cognitive interviews. Twenty-two novel items were appraised. Seven were selected for retention. This preliminary PROM-OPAC comprised 7 items to measure autonomy and was intended for administration alongside a function measure to capture meaningful acute healthcare outcomes. Development will continue with quantitative testing and validation.
Keywords
Introduction
Older people living with frailty are particularly vulnerable in the acute care setting and have poor outcomes after even short admissions (1). “Acute care” is defined as time-sensitive, unscheduled interaction for diagnosis and treatment of health problems (2). This often includes a period of “emergency care” which is uniquely characterized by uncontrolled symptoms, uncertainty, and rapid movement through care settings with multiple professionals. “Acute care” can be considered pragmatically as the first 72 h of healthcare for a new crisis, thereby encompassing concepts of community urgent care, hospital emergency care, hospital acute admissions wards, and virtual ward care (3).
Older people living with frailty receiving acute care have multiple, complex, and interacting needs (3). Existing quality measures for the acute care setting do not meaningfully capture these but instead focus on service metrics and time-based targets (4,5). Notably, there is no validated Patient-Reported Outcome Measure (PROM) that is specific for older people living with frailty who require acute care (6).
A PROM for this population and setting should measure the outcomes which users consider important. Recent qualitative research has consistently confirmed the holistic nature of acute healthcare outcome goals for older people living with frailty (7–9). Research to operationalize these goals toward measurement classified outcomes as “autonomy” (information, security, and control) and “function” (physical, psychosocial, and symptom relief) (10). A systematic review identified 4 existing PROMs potentially suitable for older people living with frailty receiving acute hospital care: the COOP/WONCA charts, EuroQol's EQ-5D-5L, the McGill Quality of Life Questionnaire – Expanded (MQoL-E), and the Palliative care Outcome Scale (PcOS) (11). In a mapping exercise, these instruments were found to adequately consider function. However, autonomy was not comprehensively measured.
This article reports the development of an instrument to measure the aspects of acute care which matter to older people living with frailty within the concept of autonomy.
Methods
Study Overview
This study sought to devise, improve, and select items with content validity to consider the autonomy theme and used methodology based on the COSMIN framework (12). Items and responses were drafted and appraised using a co-creation approach, grounding development in the user perspective. Cognitive interviews with patient participants informed item iteration and selection. The resulting items were prepared as a preliminary autonomy instrument for subsequent quantitative field testing alongside an additional function measure. A flowchart outlining the research program is presented in Figure 1.
Flowchart detailing the co-creation process. Review with lay research partners took place throughout the process and with formal consultations at each decision point (indicated by diamonds).
Lay and patient perspectives were represented throughout the study to ground item stem and response ideation, design, and evaluation in the user perspectives (13–15). Evidence was sequentially reviewed with lay representatives to guide iteration and re-evaluation.
Representation of User Perspectives (Patient and Public Involvement)
Lay Representatives
Members of the Leicester, Leicestershire and Rutland Older Persons Patient and Public Involvement (LLR PPI) Forum were engaged. This group specialized in aging-related research. Members were either living with frailty or caring for someone living with frailty and had lived experience of NHS acute care. Three members volunteered to closely support the study as “lay research partners” and were formally involved throughout research design, data collection, analysis, and reporting (16–18). Four further members supported the study's content evaluation stage as “lay collaborators”. Representatives were initially engaged in person with funded travel to meetings. During and after COVID-19 pandemic restrictions, contact was with email chains, telephone calls, videoconferences, and hybrid meetings. An action log ensured openness and accountability.
Patient Participants
Older people living with frailty were additionally represented during cognitive interviews. A purposive sample of older people living with frailty was recruited between May and August 2021. Frailty was defined using the 9-point Clinical Frailty Scale (CFS) (19). People aged over 65 years with CFS 5 to 8 (“mild frailty” to “very severe frailty”) were eligible for recruitment (20,21).
These patient participants were approached during periods waiting for assessment, treatment, or transfer at Leicester Royal Infirmary's Emergency Department. Where preferred by patients, recruitment also included their relatives, carers, or friends to help overcome barriers to participation. The approach was reparative, and thus sampling recruited sufficient participants to identify and repair problems with tested material (22).
Consent and Regulatory Approvals
Written consent was obtained from patient participants or their consultees. Consultees were approached when potential participants had capacity to express their opinions but not to consent to research, so that the broadest range of perspectives could be represented.
Development and Initial Selection
Novel items to measure autonomy were developed following a process of devising, evaluating, and selecting questions and responses (23,24). An exhaustive set of candidate item stems was produced by the researchers in collaboration with lay research partners. With permission, some items were sourced and adapted from the MQoL-E—an instrument developed for people with palliative care needs (25). Additional items were drafted de novo to consider autonomy sub-themes of the acute healthcare outcome goals framework presented in previous work (10). Transcripts from the earlier outcome goals categorization study were referred to extensively, discussed and reflected upon, and questions adapted or written to use similar language so that users might recognize concepts in the patient’s voice. Response options were devised so that the full range of reported and envisaged outcomes could be captured. Bipolar scales initially used 11 levels, anchored in both positive and negative directions using descriptive terms from the transcripts (24,26).
The candidate items then underwent reduction and iteration. This process centered on content validity appraisal and sought to retain and refine those items with best relevance, completeness, and accessibility. To minimize patient burden, at this stage candidate items were first assessed by lay collaborators using an anonymous survey. Representatives were invited to rate each item using 5-point Likert scale for relevance (not relevant to very relevant) and importance (not important to very important) to older people living with frailty receiving acute care. Optional comments were collected for each item and for representatives’ overall impression. Items with mean relevance score lower than 4 out of 5 were discarded. Comments were reviewed, organized in categories using NVivo software, discussed between researchers and lay research partners, and reflected upon to inform iterations of the remaining items (27).
Cognitive Testing
The remaining candidate items were appraised using cognitive interviews with patient participants in order to improve item understanding and interpretation (22). The approach was adapted from the “Three Step Test” (22,28). In interviews conducted by the first author, item questions were read out and displayed using a tablet device. Participants were invited to “think aloud” and share their thoughts in developing a response. Probing questions then explored the interpretation of item meaning and response rationale. Participants were asked about the relevance of each item to their own healthcare needs, the meaningfulness of themes, the ease of completion, and what they felt was missing or requiring alteration (29).
Iteration and Selection
Recorded field notes related to item relevance, accessibility, and overall comprehensiveness. Interviews were audio-recorded using an encrypted device and were professionally transcribed verbatim. Data were collected and analyzed concurrently to allow focussed re-evaluation and improvement (30). Field notes and transcripts were organized using NVivo, structuring evidence by attributes of content validity (27).
Problematic items were identified when they were misunderstood, required more assistance or time to answer, or exhibited floor or ceiling effects; such issues were iteratively addressed. Selection of items was undertaken collaboratively by the researchers and lay research partners based on evidence for content validity. The retained items were prepared as a preliminary instrument for presentation alongside the best available of previously identified function measures, EQ-5D-5L.
Results
Development and Initial Selection
The candidate items initially comprised 20 question stems and accompanying response options (Table 1, ID1). Eight items were written for the control theme, 6 for information, and 6 for security. Content was first appraised by 4 lay collaborators. Two items for control had mean relevance lower than 4 out of 5, so were discarded (Table 1, items A5 and A7). Lay collaborators’ comments informed iterations to 12 of 18 remaining items (Table 1, Action 1).
Action and Modification log Demonstrating the Development of Item Stems.
Cognitive Testing
Study Participants
Fourteen patient participants were recruited for cognitive interviews. The sample broadly represented Leicester's older population in terms of frailty, cognitive impairment, and sex distribution. Participants were living with mild to severe frailty and were aged between 65 and 92 years. Twelve were living in their own homes, 6 of whom were receiving home care. Two were living in residential care. One quarter had a diagnosis of dementia. However, recruitment failed to represent the city's ethnicity distribution: most participants were white British and used English as their first language. Demographic details are summarized in Table 2.
Summary Characteristics of Patient Participants in Cognitive Interviews.
Item Reduction
The exhaustive set was next reduced from 18 to 9 improved items during the first round of cognitive interviews (Table 1, Action 2). Four items were modified with improved phraseology at this stage, and 2 new items for the control theme were drafted. Illustrative interview excerpts which informed the action decisions are presented.
Problems were identified with longer questions, which appeared difficult to understand. Nine questions were discarded (Table 1, Action 2), predominantly relating to healthcare information and trusting others. Two items (A1 and A2) with complex sentence structures were replaced with new items considering control (C11 and C12). An information item (A10) was discarded as “situation” did not prompt participants’ focus on their reason for attendance. “Situation” had been intended to encompass all potential reasons for attendance. The term was interpreted as the consequences rather than the diagnosis itself and was modified to “health problems” in another item (A16). What does ‘situation’ mean to you? It means what can they do or are they going to do anything? Can you get better? What they’re going to give you. (Participant 4, CFS 6)
Considering security, participants were initially asked about their “usual place” to avoid confusion for people living in residential care (A15). “Usual place” was unfamiliar and difficult to interpret. This was changed to “where I live”: I feel safe in my usual place. What do you mean, my usual place? What do you interpret that as? I don’t know. Safe in my own home. I wonder if there's another term we could use? Feel safe where I live? Yeah, that's all right. (Participant 4, CFS 6)
Item B5 was discarded due to inconsistent interpretation of “health information”. Participants wanted professionals to have up-to-date records, as they did not like having to repeat their descriptions of recent events. However, “health information” was too broad and was interpreted variously from previous medical problems to current diagnosis lists and active prescriptions. Two questions on knowing test results (B7) and understanding problem severity (B9) were retained. Participants considered these information to be important: I know the results of tests and investigations I have had. They should tell you if they’ve tested you. I know it could be serious but I am waiting for results. We should be told so we know what to plan for. (Participant 6, CFS 5)
Re-Evaluation and Item Selection
Following iterative improvements, later interviews were used to re-evaluate and, where indicated, inform minor alterations to remaining items (Table 1, Action 3). Four further items with inconsistent interpretation were discarded (C1, C2, C6, and C8) and 1 item was amended (C9).
In general, participants had interpreted “feeling safe from other people” (C8) as intended, which was to identify elder abuse. However, they reported this item to be irrelevant to their situation. The underlying outcome goal was based on lay research partner recommendations rather than evidence from patient participant interviews, and therefore was less likely to be relevant for many users of the resulting PROM. Those people experiencing such problems may have felt unable to give genuine answers, and so the item could provide false reassurance that there was no problem. A modified item, “I feel safe living with my health problems” (C7), was considered relevant by all participants. Interpretation had crossover both with feeling safe in the physical home environment and also from other people: I feel safe living with my health problems. Strongly agree. I felt safe even when I was very poorly, because the staff cared and called the ambulance for me. (Participant 7, CFS 7)
Seven items measuring the autonomy sub-themes information (D1, D2, and D3), control (D4 and D5), and security (D6 and D7) were selected for inclusion in the preliminary instrument (Figure 2).
Preliminary instrument items. Seven item stems are shown with the consistent 5-level response options.
Response Scales
Patient participants found it confusing to select from 11-level scales with only end anchors. Responses were amended to use a consistent 5-level bipolar scale with all options described (“strongly agree”, “agree”, “neither agree nor disagree”, “disagree”, and “strongly disagree”). One lay research partner advised using emphasis to mitigate for barriers to concentration in the ED setting, and consequently, the end anchors were later rephrased “agree strongly” and “disagree strongly”. This scale was more agreeable: Was that too many options to choose from? No, five is about right. (Participant 7, CFS 7)
The reference period was also modified following early interviews. A shorter period (“today”) prompted focus on users’ current acute care situation. It was reasoned that this would also improve sensitivity to change during recovery or deterioration.
Discussion
This article presents the qualitative phase of preliminary instrument development to measure the autonomy aspects of what matters most to older people living with frailty receiving acute healthcare (10). The research built upon existing literature and extensively engaged lay and patient representatives to produce a preliminary measure comprising 7 novel items. These will be quantitatively field tested in future work alongside EQ-5D-5L as a complementary measure of function.
These 7 novel items consider one's sense of safety, knowledge of health information, and opportunity to participate in care decisions. While there is an apparent conceptual distinction between the items for information (knowing one's results, prognosis, and next steps), control (feeling involved in decisions), and security (feeling safe and supported), the items are interrelated as meaningful sub-themes of autonomy in this setting (10). Biographical consequences of living with frailty and the need to consider self-determination and uncertainty are addressed (31,32). While these are no doubt important to any recipient of acute healthcare, as practice principles they represent the core of geriatric emergency medicine, wherein the approach is person-centered and holistic for people whose complexity is not addressed by common single-problem care models (3,33). Many of the excluded items considered specific scenario outcomes, including prescription changes, healthcare records, treatment plans, and elder abuse. The 7 retained items instead consider more generic outcomes with apparent relevance to a heterogeneous population. The relatedness of items forming the proposed structure requires examination with factor analysis during subsequent phases.
Implications for Research
This study's parallel workstream identified EQ-5D-5L as a suitable measure of function, mapping to those sub-themes of acute healthcare outcome goals (11). The EQ-5D is a generic measure of health-related quality of life and was found not to consider some outcomes specific to older people living with frailty receiving acute care. Previously in other specific settings, issues with the comprehensiveness of EQ-5D have required the use of “bolt-on” measures (34). The feasibility of administering EQ-5D alongside this novel autonomy measure in the acute care setting requires evaluation during field testing.
Selection of autonomy items provoked tension between the depth of detail (which inevitably requires a longer instrument) and the accessibility afforded by a shorter measure. The benefit and burden to unwell older people participating in PROM collection in busy acute care settings will be further evaluated during the applied phase of this research program.
The timing of PROM administration is likely to influence the success of eventual program implementation and requires exploration during validation studies. In the current study, participants were often stressed and frightened during the earlier period of their hospital attendance. Some preferred to participate with a friend or relative present; these “patient partners” provided reassurance or assistance and did not respond themselves as proxies. Staff involved in collecting instruments will need to be sympathetic to people's ability to reflect on their situation and desire to report this while unwell.
The optimal format for PROM collection from older people requires investigation in further research. Older people with lower literacy, advancing age, and cognitive impairment have had difficulty using the COOP/WONCA charts and EQ-5D in paper form (35,36). However, their ability to access computer technologies has been demonstrated, especially when they are consulted during design (37). Electronic collection systems can improve accessibility, including with color and size adjustment. The effect of electronic-aided administration on PROM completion by older people in acute settings requires evaluation.
Strengths and Limitations
Lay representatives, including people living with frailty, were engaged to a greater extent than has often been possible in this field (38). Indeed, previous PROM development studies have rarely engaged end users as research partners (39,40). In this study, lay research partners were extensively involved in decision making throughout the co-creation process, providing assurance about the instrument's meaningfulness. Partners had substantial experience advising on other aging-related research through the LLR PPI Forum. However, they did not have the expertise or specific training in clinical psychometrics, and therefore ongoing validation work is testing the psychometric properties of the preliminary PROM.
Cognitive interviews were conducted by a single researcher who had also coordinated the novel item drafting and existing instrument identification, representing a source of potential bias. This was mitigated by the involvement of lay research partners and other researchers, including individuals with psychometric expertise. Additional studies are planned for the evaluation of content validity, feasibility, and implementation factors from the perspective of healthcare professionals.
Patient participants were categorized by their sex as listed in healthcare records; future research investigating social constructs should instead consider reporting the gender distribution. The sample's representation of people with non-white ethnicity was disappointing: there were limited opportunities to engage consultees during COVID-19 restrictions and healthcare-seeking behaviors were likely to have been unusual during the recruitment period. This has prompted reflection and action among the researchers, including training in cultural competence and recruitment diversity, and consulting a local center for ethnic health research. Nonetheless, the candidate items were devised following research with a diverse and analytically rich sample and were judged here to have promising content validity.(10) Assessment of external validity will begin during field testing with a larger patient cohort.
Conclusion
Seven novel autonomy items were co-created with lay and patient representatives. Field testing and validation of this preliminary instrument, “PROM-OPAC”, alongside EQ-5D-5L as a complementary function measure are ongoing.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the BMA Foundation for Medical Research (Lift Into Research grant) and the National Institute for Health and Care Research (Doctoral Research Fellowship NIHR300901).
Ethical Approval
This study was approved by the West Midlands & Coventry Research Ethics Committee (reference 19/WM/0098).
Statement of Human and Animal Rights
All procedures in this study were conducted in accordance with the University of Leicester's approved protocols.
Statement of Informed Consent
Written informed consent was obtained from patients or their personally authorized consultees for their anonymized information to be published in this article.
