Abstract
Previous reviews of experience-based codesign (EBCD) projects have shown non-participant observations (NPOs) are omitted or unreported in the majority of studies, despite academics’ insistence of their value. We undertook Stage 1 of an EBCD project using a Case Study Research approach to organise the inquiry. NPOs were used as the first object (frame of analysis) of the case subject, exploring and understanding the experience of users and providers of ED care. 162 hours of NPOs were conducted by six research team members across three hospital Emergency Departments (EDs), representing a local nested case. The four principles of EBCD were used to guide the inquiry, providing rich understanding and description of the context, and valuable insights into critical issues. Summarised NPO narratives allowed the research team to familiarise themselves with the different physical environments, workflows, and processes, as well as think more deeply about work and interactions. Conducted within complex healthcare settings, this example demonstrates the value of observing Work-As-Done and has advanced knowledge of institutional logic that will be critical in considering realistic and sustainable change initiatives. Commitment to intentional design and well-reported research methods for exploring and understanding lived experience can satisfy academic audiences of the study integrity and provide comprehensive information for those undertaking accelerated EBCD.
Introduction
Non-participant observations (NPOs) of the interface of care are an important step in an experience-based codesign (EBCD) approach to health system improvement (Bate & Robert, 2007). EBCD is a two-stage participatory approach that co-opts those with lived experience of receiving health care with those providing health care to help design more user-centred outcomes (Francis-Auton et al., 2024; Masterson et al., 2022). In the first stage, the aim is to understand in depth, the experience of users and providers of the service through NPOs and interviews with users and providers. Developing an understanding of work-as-done at the frontlines of healthcare is crucial in developing useable and effective solutions to improve care delivery in complex systems (Braithwaite et al., 2016; Hollnagel, 2017). The purpose of NPOs is for non-participants to immerse themselves in the real world of healthcare delivery, develop emic understanding of the care interface, empathise with both the provider (often staff) and user (often patients), and reflect on their individual perspectives as an objective inquirer (Bate & Robert, 2007; Green et al., 2020). Previous reviews of EBCD projects have shown NPOs tend to be omitted or under-reported (Donetto et al., 2014; Francis-Auton et al., 2024; Green et al., 2020); only 12 of 64 papers included in a 2023 review of EBCD peer-reviewed literature included NPOs, ranging from 2 to 219 hours (Francis-Auton et al., 2024). Description of how NPOs were undertaken or impacted the non-participant standpoint were unclear (Francis-Auton et al., 2024), and few studies presented the service provider perspective (Green et al., 2020). We asserted that omission or disregard of key steps in a methodology may undermine the integrity of the approach (Francis-Auton et al., 2024). Given the premised importance of NPOs to understanding and representing user and provider experience (Bate & Robert, 2006), failure to properly prepare may also infer an injustice to participants who generously share their lived experience and contribute time and effort to the codesign. More clarity about the process of NPOs and the insights they can provide may guide informed application in EBCD approaches to healthcare improvement.
Observations are used as a qualitative research tool so the researcher can notice particulars about a field setting, aligned with the research purpose. Cresswell (2013) lists four ways the researcher may participate - completely, fully engaging with the people being observed; as a participant, where the observer is participating in the site activities; as a complete observer, unnoticed by the group under study; and as a non-participant observer, an outsider of the group, watching and taking field notes at a distance (Cresswell, 2013). While NPOs draw from ethnographic research paradigms, they are not intended to be an in-depth observation of people and culture. Rather, they allow observers to gain a privileged view of a specialist environment that may otherwise have only been imagined. Areas of interest may include work processes and environmental factors that enable or constrain work, or interactions between staff and patients. In EBCD, the premise is that rich awareness of the context enhances understanding for subsequent analyses and codesign (Donetto et al., 2014; Green et al., 2020), and description may help readers determine the extent to which research findings may be transferrable to other health system settings (Francis-Auton et al., 2024). Given the proposed value of NPOs, but limited exemplars in peer-reviewed literature, there is a clear need for explication of methods and insights to advance epistemology.
An Example of NPOs in Emergency Departments
We undertook NPOs as the first step in an EBCD approach to identify and design new or adapted models of Emergency Department (ED) care (Figure 1) (Cheek et al., 2023). Diagram of our staged 5-year Experience-Based Codesign project, where a Case Study Research approach was used to accomplish Stage 1, Exploring and Understanding the experience of users and providers. Updated from: (Cheek et al., 2023).
The first stage of this EBCD was undertaken across three sites, with multiple patient cohorts. As we were undertaking this EBCD project as research, we drew from a case study research (CSR) approach, where we sought rich description of a contemporary, real phenomenon through multiple sources of data and multiple methods (Cheek et al., 2018). The context of our study was Emergency Department care in Australia.
Emergency Departments in Australia
EDs, as a sub-system of the broader integrated health system, are purpose-built to provide 24-h access to rapid assessment, treatment and/or stabilisation, and referral to hospital inpatient or community-based care (AIHW, 2022) (Figure 2). Drivers influencing demand for ED care.
ED care in Australia is provided free to residents, EDs are never closed, and rarely, if ever, turn away patients seeking care. In our study setting (as well as globally), EDs have featured increasingly in public news articles, portraying problems that allude to unsafe and delayed care (Austin et al., 2023), and hospital ‘ramping’ (where paramedics cannot handover ambulance patients due to a lack of capacity within the ED). In reality, more people are seeking emergency care (AIHW, 2022), and those with the most urgent needs are generally seen on time (AIHW, 2022).
The scope of ED care has also been increasing in response to insufficient resources elsewhere in the system. For example, a lack of community mental health services (Austin, Cheek, et al., 2024) and difficulty accessing primary care (Morley et al., 2018) are two of a number of drivers of ED presentations (Figure 1). At the same time, hospitals are experiencing difficulty discharging inpatients to required high care services such as residential aged or disability care, deferring definitive care for patients still in ED or requiring EDs to extend their care role beyond the initial assessment and referral phase. Overcrowding in ED and access block (>8-h delay transitioning a patient to an inpatient bed) (ACEM, 2022) also impedes timely offload of ambulance patients. These systemic issues are reflected in quality care indicators; for example, delayed care or prolonged ED length of stay are factors associated with poor patient outcomes and negatively impact patient and staff experience (Bernstein et al., 2009; Carr et al., 2007; Gaieski et al., 2017; Peltan et al., 2019; Richardson, 2006). The COVID-19 pandemic put additional strain on EDs, where rigorous infection prevention and control measures were necessarily implemented, while maintaining core business function (O'Reilly et al., 2021). While workforce shortages were being experienced in EDs prior to COVID-19, higher levels of exhaustion and burnout post-COVID 19 are thought to have contributed to further workforce shortages (Dixon et al., 2022; McCormick et al., 2023). Nevertheless, EDs are committed to improving care delivery to continue to meet the needs of the communities they serve.
Improvement initiatives are based on the premise that change will enhance the quality, efficiency, and safety of care, but scaling up and spreading change initiatives in healthcare has proven difficult to achieve and sustain (Greenhalgh & Papoutsi, 2019). A challenge of health service research is that it is often focussed on outcomes that are restricted to meta-level analysis through macro-level process, rather than actions at the micro and meso levels (Zilber, 2013). For example, ED length of stay, aggregated retrospectively from system data, is an important outcome measure for hospitals as it is associated with politically-driven incentives and patient outcomes (also aggregated and measured retrospectively), and is used as a performance comparator across organisations. However, constraints at the department level such as staff skill mix or availability of supporting diagnostic services may not be evident or accounted for in these outcome measures. Subsequently, improvements found to be successful in one ED (through meta-level outcome measures), may not produce the same outcome result in another ED (Testa et al., 2024), as important contextual factors, including variances in the settings and social interactions, may impact the efficacy of similar interventions (Ansmann et al., 2020). Additionally, initiatives prioritised by senior managers or hospital executive have been found more likely to be successful at a department level (Clay-Williams et al., 2020). Understanding context is then a fundamental aspect of EBCD and must be accounted for when identifying and prioritising improvement interventions in complex and multiple healthcare settings. In this paper, we aim to report the findings of NPOs undertaken in three hospital EDs as an example of method, but further, to illustrate their value in understanding the context of care and implications for subsequent codesign work.
Methods
ED – MyHospitals 2022-23, Sources: (Australian Institute of Health and Welfare AIHW, 2022).
ATS – Australasian Triage Scale (ACEM, 2022).
Local Community
At over almost 1 million people in 2022, the health district population has been growing annually and is culturally and socio-economically diverse: 49.8% of people were born overseas, and 54.1% of households speak a language other than English at home; the number of people with a Bachelor degree or above and the average median weekly personal income are both higher than the state average, but the unemployment rate is also higher; the district contains 13.9% of the state’s social housing (21,036 dwellings), however these dwellings are concentrated in some local areas, while other local areas are socio-economically advantaged (ABS, 2022). While 34% of the district population is under 25 years old, 4.7% are ≥65 years, and 1.4% of the population identify as Indigenous Australians (ABS, 2022).
Situating the MyED Research Team
MyED Research Team (as of February 2024) Professional Backgrounds.
ŧ Professional = paid employment; *Self-reported M-Male, F-Female, NB-Non-binary.
Ethical and Governance Considerations
The MyED project obtained ethics approval from the Local Health District Higher Research Ethics Committee (REF: 2022/PID02749-2022/ETH02447). The methods followed the protocol described in full elsewhere (Cheek et al., 2023). Following ethics approval, all research team members who were not employed by the health district obtained contingent worker status to allow site visitation. This comprised health screening, criminal history clearance, vaccination status and updates, and induction training such as safety, privacy, and conduct standards.
Each observer was assigned a senior staff member as a ‘buddy’. The buddy was a touchpoint for the observer – checking that the correct Personal Protective Equipment (PPE) was used, knowing where to take a break, checking in if there was a significant event. For each shift where researchers conducted observations, the clinician conducting the start-of-shift briefing informed all ED staff present of the researchers observing the shift to understand process and flow; the staff were informed that if the researchers wished to collect any notes about an observed personal interaction between staff members, they would seek written consent. During the shift, observers presented themselves and the study purpose openly and honestly to staff and patients. Observers took care to remain in the general work area and did not shadow any staff member to the patient bedside. As no direct patient care, interactions of a personal nature between patients and staff, nor any personal health information were subject to observation, written patient consent was not obtained. Instead, the observer carried an approved short plain-language description explaining their presence in the ED area for any patients or staff inquiries.
Data Collection
Clinical research team members and ED Nurse Unit Managers nominated shifts suitable for research team members to attend sites, so that a broad range of ED activities could be observed. At mutually agreed times, research team members conducted observation shifts at all three EDs; observations were carried out on various weekend, weekday, morning, evening, and overnight shifts so that we could study the continuous institutional work being undertaken by various staff to accomplish every day, practical work. Research team members attended for shorter periods – 4–8 hours, and for longer 10–12-h shifts. While conducting observations for the longer period is demanding, as ED staff work these lengthy shifts, it was important for the team to understand the level of fatigue experienced (emic understanding and empathy). The team leader whose main role was to maintain patient flow in the ED (rather than provide direct patient care) was the role most suited for research team members to shadow to observe workflow and process and avoid direct patient care. Other roles observed included clinical nurse unit manager (CNUM) – overseeing the organisation of patient care and/or staffing, and navigator – organising discharge of patients from the ED. Written consent was obtained from all individuals shadowed, as they interacted from time to time with the observer, explaining their work.
Observation and familiarisation with ED work processes and flow was the core aim and was emphasised with researchers prior to their shifts. Observation notes were not taken in a prescriptive format; we used the domains of the Work Domain Analysis (Austin et al., 2024; Naikar et al., 2005) to guide non-participant thinking around purpose, objects, and processes (Appendix 1), but also encouraged research team members to make other notes of their choosing.
Data Analysis
Researchers discussed their NPO experience among the team in a weekly meeting, checking understanding of processes, clarifying uncertainties, and reflecting on preconceived ideas about ED work and how these had been affirmed or challenged. At the completion of all NPOs, team members CC, RCW, EA, NR, MS, LR, and LT individually reviewed all the observation notes generated by each member and determined individual impressions. Team members met together in a half-day workshop to share their individual impressions of the reviewed observation notes. The workshop discussion began by acknowledging the shift routines, common processes and structures, interactions and how staff responded to different care demands and contingencies. We then discussed the differences observed between settings, staff, and the influences of interfacing actors or services; that is, the social and systemic challenges to providing care in the ED that might impact each ED differently. The summary observations (reported below) were checked with clinical members of the MyED project team (MM, AD, MV) and with the project Steering Committee (which includes user representatives) to validate the academic team member’s interpretation of observations.
Results
Non-participant Observations Conducted at MyED Emergency Departments.
General Observations
EDs are very noisy and busy environments: the nature of the work and the presenting behaviour of patients can be confronting for the new observer. Although each ED had a different physical layout and size, they were each comprised of the same compartmentalised areas that aligned with models of care: a patient waiting room for those arriving by their own transport, a triage and assessment area, a fast-track area for patients whose care could likely be delivered quickly then discharged, a short stay area for patients who do not require inpatient admission but require further investigations and are likely to be discharged within 24 hours, and acute beds for those requiring longer stays, including admission to a hospital bed. An arrival point for ambulance patients included a triaging area and adjacent designated resuscitation bays for the most critically ill or injured patients.
People presenting to ED across the three EDs represented the whole of society – children and geriatrics, people living at home independently, from supported group living, from custodial sentences in the justice system, the able and those living with disability, with physical health or mental health issues. Depending on the individual’s circumstances, they came alone, with a friend or family member, a formal or informal carer, escorted by paramedics, or in the custody of police. Some waited patiently, some became agitated, some slept. The cohorts most noticeable in the patient mix were older adults, people from a culturally and linguistically diverse (CALD) background, and those with mental illness. EDs are necessarily quite open environments so that staff can clearly observe patients and monitor for clinical deterioration. This meant patients were also immersed in the diverse patient population and their behaviour, which seemed to be confronting for them at times.
The three observation categories presented below, illustrated with excerpts from NPO notes (where [SM] denotes a Staff Member), demonstrate insights gained from observations that will impact codesigned improvement initiatives.
Differences Between the EDs
The three EDs were all structurally different, operated under different conditions, with different constraints:
ED1 was a metropolitan district ED which offered 24-h emergency care but admitting capacity was limited to paediatrics and palliative care. Other inpatient admissions from ED1 were transported to ED2 once there was a bed available. This involved communication with ED2 and a liaison with a hospital transport service once a bed was allocated. ‘Mid-afternoon, beds are allocated to all five patients awaiting transfer – the [SM] is pleasantly surprised and immediately books transport. Transport provides an ETA [Expected Time of Arrival], but this changes periodically; by the end of the night, the ETA has changed by 8 hours for one patient. With each update, the [SM] goes to the bedside and informs the patient.’ (NPO 2, weekend afternoon shift, hospital 1)
During the night, support services to ED1 were limited – pathology was couriered to ED2 every few hours, ED nurses must escort and stay with patients needing radiological imaging, and an ED doctor must accompany a patient requiring a CT-scan with radiological contrast. One doctor and one nurse contributed to a 24-h Code Blue (medical emergency) team that responded to calls for urgent medical assistance from anywhere on the hospital campus (during the month immediately preceding observations, there were 25 Code Blue calls). When called away, the remaining staff absorbed their workload. ED1 and ED2 were governed as one site.
ED2 was part of a new major metropolitan facility and staff were still assimilating with their new space. Staff reported that in the sixth months prior to NPOs there had been a lot of senior nursing staff resignations, so nursing staff juggled shortages in some positions with a more junior skill mix that included a large proportion of casual or agency nurses. Both paediatric and adult patients attended the ED, but there were different care areas for each.
ED3 was an adult only ED (a paediatric only ED also operated within the same hospital campus). The hospital was a major referral hospital for the region, providing all major clinical specialties through different wards and clinics, supported by 24-h services. A 4-bed mental health pod had been recently established quarantining existing clinical bed spaces for patients presenting with mental illness. Staffing appointments appeared to be more stable than ED2.
The EDs work Overcapacity Every Day
At the time of data collection there were staff shortages, however member-checking months after the observations found these had largely been resolved, albeit leaving a more junior skill mix. The main continuing issue was the overwhelming number of people presenting to ED, with ongoing access block. All three EDs were full every day the NPOs were conducted, with people waiting to be seen due to the number of people presenting for care. ‘There’re two ambulances waiting. There’re no beds. It’s crowded in the ambulance corridor. [SM] is now working here to find beds. They walk over to the bed area and chat with a nurse to find beds. They do this twice. There is a screaming adult man in the paediatric emergency room. [SM] tells me the short stay beds are full. They’ve used up the paediatric emergency beds. [SM] tells me if a bat [critical patient inbound] call comes in, they’ll just have to figure it out then,’ (NPO 4, day shift, ED1). ‘Start of day in ED and is ED full, its 6.46am. Patients in waiting room for ∼20 hours, 20 in waiting room. 6-8 ambos, 1 ambo ramped, 1 staff member down, 5 waiting to go to Operating Theatre. After hours coordinator pressuring [SM] to fill last resus bed – [SM] says OK, but tells staff they don’t want to fill it,’ [in case they get patient in requiring urgent resus] (NPO 2, day shift, ED2). ‘1:50pm Busy Around 6-8 paramedics in the hallway for the last 30/60 min. Ambulance beds with people in the hallway. Everyone is looking for empty beds. 2:10pm still struggles finding empty bed. There are doctors asking what they can do/who they can help. They can't move the 8 patients behind triage because other MOC are full too. Patient somewhere ready to be moved (create an empty bed) but can't be moved because the bed is broken, ‘(NPO 5, ED3).
On arrival to the ED, patients registered with the clerk, creating a record of their visit in the hospital information system, with complete and accurate personal details. They were then almost immediately triaged by a nurse to assess the clinical urgency of their presenting complaint; those requiring very urgent attention were moved directly into clinical care, while those requiring less urgent attention waited until clinical spaces became available. ‘Patients can wait outside behind the fire hydrant and staff look after them; there were no chairs outside, but I’ve heard they put chairs out throughout the day when the ED is overcrowded. [Older lady in wheelchair] patient said she’d be outside waiting if it´s ok and the nurse replied “we´ll find you”,’ (NPO 6, day shift ED2).
The first task of team leaders coming onto shift was to familiarise themselves with all the patients in the department, including those in the waiting room. To manage patient flow, the team leaders sought to progress their care and make room for expected patient influx. Thus, for the whole of their shift, in addition to being mindful of all patients in allocated clinical spaces, they maintained overview of all patients in the waiting room. ‘[SM] says that the sickest [patients] often self-present, but this then biases their dispositioning towards the waiting room - they only get to come in if they are sick enough to bounce another patient from an acute bed. And by then they can be very sick (it’s not as noticeable while they are in the waiting room as on the ambulance trolleys as [they’re] not continuously monitored)’ (NPO 1, day shift ED3)
As well as those patients who arrived by car, ambulances transported patients through a separate entrance, and those not needing immediate resuscitation were offloaded into an acute care space. Until the two paramedics attending the patient formally handover the care of the patient to the ED nurse, the paramedics must stay with the patient, which prevents them from returning their ambulance to community response. In the public domain, this is commonly termed ‘ramping’. However, in order to offload the patient, there must be an available clinical space. Staff attempt to allocate clinical space to those who need it most, however staff are under considerable pressure to prioritise ambulance patients (and receive phone texts from hospital executive) so the ambulance can respond to other emergency calls from the community. ‘There’s lots of push from the hospital executive to offload or prioritise ambulance patients over waiting room patients. The team leader has to have strong views to stick to clinical decisions about whether or which patients to bring into the department.’ (NPO 3, ED3)
When there were no clinical spaces available and the ambulance patient was able to ambulate and was assessed to have a low acuity problem, the patient might be transferred to the waiting room to wait with other lower acuity patients. Older lower acuity patients were rarely offloaded into the waiting room, however.
The third way patients enter ED is through referral from within the hospital; for example, an unwell visitor or outpatient, or someone who had a medical emergency in other areas of the hospital.
Presenting patients who were waiting to be triaged caused considerable concern for staff as they worried that there might be an as-yet-unidentified acutely ill patient needing urgent care.
With Busyness, Focus Moves From Goals (Provide Care) to Tasks/Processes (Complete These Steps)
At about 10am each day, an influx of patients arrived by car seeking ED care; in one ED this meant there was a queue of 20 or more patients waiting to register. As the work in the ED increased, staff became more focussed on tasks and process, centred on the information system – recording patient observations, assessments, reading test results and the plan of care, and shuffling patients to the next designated care area to free up assessment space. ‘Ambos started around 9.30 – its 1010 and 6/7 ambo bays full. Nurse having to redo preop checklist as not saved in new system. Trying to find old form,’ (NPO 2, afternoon shift, ED2). ‘[SM] logging into the computer says, “could triage someone in this time”. It’s a swipe in system, but computer response is slow in terms of ED pace - slow to respond to log in, and slow to change screens or recognise keystrokes. Only a few seconds and would not be noticed by someone in an office but too slow for ED,’ (NPO 3, day shift, ED3).
Patients sometimes perceived the lack of attention as being forgotten about or neglected, and some seemed unaware that ED prioritised the clinical need of patients ahead of chronological order of presentation. Staff directed us to hospital Google reviews for patient perceptions: ‘Very disappointed…, very bad customer services, I came tonight regards by chest pain and stomach pain. They told me it’s quite long wait, I was waiting from 8pm then I saw new people coming and getting in before us.’ Google Reviewer 1 ‘Useless! Even if you’re dying, you need to wait a minimum of 4 hours. All staff is not interested, they see it as a job.’ Google Reviewer 2
Staff usually work 8–12-h shifts. The fatigue level at the end of 10- or 12-h shifts, particularly on night shift was felt acutely by researchers. ‘[SM] back round at ambulance triage. Lot of time on feet (almost no sitting so far today – turns out there will be no sitting all day and I go home with very tired and achy feet!’. (NPO 3, day shift ED3) ‘6.30am. I am super weary. [SM] is going from person to person, trying to solve [their] shift planning problem. A family who arrived around midnight have just got a paediatrics bed.’ (NPO 1, night shift ED1)
The continuous busyness of the ED was surprising to most team members, as was the level of fatigue experienced toward the end of a long shift. Team members realised that many quality indicators of ED care, for example length of stay, while largely attributed to the ED, were beyond the control of ED staff as they were dependent on many other external variables such as availability of beds, transport, time to imaging and diagnostic tests. While aware of system performance indicators, frontline clinicians were more directly concerned with patient wellbeing; indicators such as length of stay were articulated as negatively impacting the quality and safety of patient care (patient harm was associated with personal culpability and risk to professional registration).
Reflecting on our observations, we perceived there were likely to be diverse patient experiences; for example, those patients who needed urgent care may be more satisfied as they would have received this in a timely manner. In contrast, those needing less urgent care (in the context of urgency of the ED workload, not the perception of the patient) may be unsatisfied as they probably would have had to wait hours to be seen.
Discussion
This paper provides an example of NPOs conducted in three EDs to advance knowledge about methods and the value derived by a research team for subsequent steps in an EBCD approach. We drew on a CSR approach to the first stage of this EBCD project and used NPOs as a frame of analysis (object) applied to the subject of user and provider experience of ED care. Our observations helped in visualising and articulating the differences between the three EDS – the spatial layout, team dynamics, and processes, and hidden contextual elements important in situating staff and patient perspectives derived from subsequent interviews exploring experiences. NPOs provided the research team with a privileged understanding of the demanding environment in which ED healthcare providers work, the range of experiences patients may encounter seeking ED care for their perceived urgent need, and the external circumstances that constrain how EDs might adapt.
NPOs provided a more complete picture of work-as-done (Hollnagel, 2017), with connection to a wider range of stakeholders than the preferenced ‘patient’ as user and ‘ED staff’ as provider. Observing providers work is as important as asking them about their practice as those who do the work can be unaware of the effects and implications of their interactions with other workers on system behaviour. In complex systems such as EDs, it is often the interpersonal interactions, rather than the individual behaviours, that have the greatest effect on system performance (Hollnagel & Braithwaite, 2019). Previous reviews have affirmed the importance of NPOs in providing rich insights into how and why things work and lamented the lack of NPOs in reported EBCD literature (Donetto et al., 2014; Francis-Auton et al., 2024; Green et al., 2020). We also propose that NPOs were critical in understanding the institutional logic that maintains the rigid character of the healthcare organisation. Framing the system in this way was important in thinking more deeply about the relationships between the system functions, external constraints and expectations, and the implications for improvement initiatives.
Methodological Insights
The ED environment was very new to some research team members. Note writing was sometimes a distraction for these members and it proved more valuable for them to talk with other research team members soon after their observations to help make sense of their experience, rather than take notes about parts of work process which was unknown to them. This allowed the unknown to become known. As the researchers could not attend all at once, different shifts offered different experiences and observations. The diverse range of researcher backgrounds meant shared perspectives of observations provided a richer picture of the environment for all. Through conducting NPOs, researchers found they were able to build trust and greater empathy with the work of providers and the experience of users.
If EBCD is approached as research, rather than quality improvement (QI), it is imperative that it is well described to demonstrate intentional design and justified research methods. We found the use of a CSR approach useful in organising the inquiry in the first stage of EBCD, providing a flexible research framework for the use of multiple methods and objects of analysis in a complex social situation (Cheek et al., 2018). Those undertaking Accelerated EBCD projects (AEBCD) rely on experience collected and reported by others to move to codesign more quickly; well-reported Stage 1 methods using a CSR approach can provide assurance that the claims being made are supported by study integrity and allow judgement about the transferability of findings to another setting. We used a Work Domain Analysis framework to stimulate thinking around system purpose, values, functions, processes, and objects. There are other tools used in systemic design that may be used to guide observations, such as stakeholder analysis, or process mapping. On the surface, observation notes may seem very subjective snapshots, definitely influenced by researcher axiology and the type of role the observer shadowed. However, in reviewing the collective observations about the ED, consistent with a CSR approach, we were able to think more deeply about the social and systemic truths of the real world of healthcare. Moreover, as we proceeded to interviews with providers and users, we drew on our NPOs to make sense of user and provider experiences. For example, in one interview a user commented that they understood the staff were busy as a nurse had told them a bus had just arrived. From our NPOs, we knew this metaphor was used by staff to describe the phenomenon of a crowd of people presenting individually to ED at around 10–11am, resulting in a queue of people registering for care. We also used NPOs to triangulate with interview data to develop touchpoints, the output of EBCD stage 1, to move to stage 2, the codesign itself (Appendix 2).
Advancing Institutional Understanding of Work-As-Done
Health systems are complex; they comprise functionally independent sociotechnical sub-systems, clustered and/or geographically distributed, that interact with one another in various ways to deliver healthcare to individuals and communities across the care continuum. As the EDs we observed were working over capacity every day observations were undertaken, understanding the system logic will be crucial in modifying institutional work.
Health systems exist within an institutional environment in that they do not have output that is immediately identifiable (Andersson & Gadolin, 2020). That is, compared with technological systems that produce easily identifiable products (such as a smart watch or an electric vehicle), health systems’ more ambiguous activities and technologies are directed toward providing patient care, the quality of which may not be immediately identifiable. Instead, legitimacy is sought through identifiable structures and activities such as performance indicators and quality assurance processes. We observed that many key performance criteria for ED care were beyond the control of those providing care, and that lack of agency was associated with staff distress. Quality conformance is also subject to external expectations and regulations through law, funding, health professional standards, unionism, quality assurance standards, accreditation, social norms, and political directives. Institutional theorists maintain the environment determines the options and limits discretion in the choices available for health system leaders. In institutional theory, these expectations and pressures on the institutional organisation can become a cage that compels them toward the implementation of actions and strategies that resemble others in the field, particularly under conditions of uncertainty (Powell & DiMaggio, 2023). Thus, the functional complexity of any codesigned initiatives must be tailored to and provide sufficient options for interacting with the known and potential of the ED environment.
Effective work design is directly related to accurate analysis of requirements (Jones, 2014). NPOs provided critical insights into the social and technical needs of ED. Failure to observe the way work is undertaken and the values and principles of those caring for patients, as well as the external expectations and pressures acting upon them, was illustrated in the use of the ED digital health record. The drive to computerise work, often with the misperception that computerisation in and of itself will deliver benefit, has resulted in many electronic health record initiatives that have changed, but not necessarily improved, the work environment (Miller et al., 2015). While the digital health record in ED provided a central and shared repository to view information about all patients in the department, work to manage overcapacity was organised around computer interfaces rather than patients, and the interface was sometimes slow. Patients sometimes misconstrued this alternate focus as staff not caring about them. For staff, there can be a problem if the available information is relied upon to provide an accurate representation of the situation, as this may not always be the case. Actors further removed from the ED may also make judgements or decisions based on the available information that have little foundation in what people are currently facing on the ground. This was exemplified by the prompts from executive to staff phones to offload ambulances based solely on the ambulance arrival time; at times there was no space to offload them, the staff perceived another patient had more urgent clinical need, they were attending to an urgent situation, or they were trying to triage an influx of patients arriving by car (who may have been sicker and were unattended). Conflict occurred for ED staff when they experienced a tension between what they were being directed to do and what their professional obligation and clinical judgement dictated. That is, there was a conflict between professional logic (related to patient care), and managerial logic (related to bureaucracy and external expectations) (Andersson & Gadolin, 2020). During our NPOs, the strength of the ‘home’ logics of ED and those of interfacing services including ambulances and management, and the tension between these logics, became most evident when the institutional arrangements were most challenged; that is, when EDs were working over capacity. Actions and interactions at the micro level, and power distribution across the organisation and beyond, highlighted how change initiatives may (or not) be successful, in different situations and ED settings, influenced by external expectations. Understanding the structures, tensions and inter-relationships that influence behaviour and change conditions is therefore critical in selecting design initiatives that will create value and are likely to be supported, including re-interpreting institutional signals to reinstate agency (Andersson & Gadolin, 2020) and re-creating meaningful work at the micro level (Larsson et al., 2023).
Implications for further steps in the EBCD study
Frontline clinicians do not usually undertake NPOs as it is difficult for them to assume the role of a naïve observer and they are time-poor; patients do not usually undertake NPOs due to governance and privacy implications around the care interface. Thus, it is the role of researchers to orientate both to the human life world of the other as an introduction to codesign. Traditionally, filmed interviews were used for this purpose, but ethical and moral implications and methodological challenges point to the need for alternatives (Francis-Auton et al., 2024). The researchers’ understanding, having undertaken NPOs, can shape personas or hypothetical situations that may bring to life an aspect of care from both perspectives.
Strengths and Limitations
There is no prescribed quantity of NPOs; we found completing day and afternoon/night as well as weekend shifts, and at least one shift at each ED, critical for understanding the unique characteristics of care delivery for that ED. Not all researchers stayed for the full 12-h shift that the ED staff typically worked, but the more closely aligned with ‘work’ (i.e., the work shift that staff do) the more deeply researchers were able to get insights into care delivery experience. Deciding the volume of NPOs will likely depend on the researchers existing level of experience with the system of interest, the diversity of work performed in the system, and the variations in work over time. The close nature of observations may have changed the way work was undertaken, however the observations by different individuals at different times and days enhanced the trustworthiness of insights gained. The team leader, clinical nurse unit manager and navigator roles were selected by the clinician team members as the role to follow for NPOs as it was the least obtrusive on patient privacy and direct care, and allowed an overview of flow, process and interaction through most of the department. This may mean that other potentially important elements of the ED context and day to day operational reality may not have been captured in the NPOs reported.
Conclusions
Overall, the NPOs did not overburden the research team (albeit well-resourced for this project) and delivered crucial insight into the social and systemic challenges that will be important in codesigning high-value, sustainable, care improvement initiatives. A CSR approach was a useful research framework for organising Stage 1 of an EBCD project. NPOs enabled research team members to challenge assumptions, critically analyse issues and reference the particular. NPOs also provided a mooring for understanding user and provider experience and sense-making.
Supplemental Material
Supplemental Material - Non-Participant Observations in Experience-Based Codesign: An Example Method and Value in Exploring Emergency Department Care
Supplemental Material for Non-Participant Observations in Experience-Based Codesign: An Example Method and Value in Exploring Emergency Department Care by Colleen Cheek, Elizabeth Austin, Lieke Richardson, Luke Testa, Natalia Ransolin, Emilie Francis-Auton, Mariam Safi, Margaret Murphy, Aaron De Los Santos, Matthew Vukasovic, and Robyn Clay-Williams in International Journal of Qualitative Methods
Footnotes
Acknowledgements
We would like to acknowledge the MyED project partners: the Australian Institute of Health Innovation at Macquarie University, the Western Sydney Local Health District (WSLHD), the University of the Sunshine Coast, the University of New South Wales, the NSW Agency for Clinical Innovation (ACI), the Department of Social Services (DSS), the NDIS Quality and Safeguards Commission and Health Consumers NSW. We acknowledge the Australian Government Medical Research Future Fund (2022:APP2018361) who provided us with support to establish the MyED collaborative project, and consumer and clinical representatives for their role in providing expert advice and input.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Authors MM, AS and MV are employed at the study EDs. As such they did not participate in or influence analysis of observations, but member-checked the summary observations as stated in the manuscript.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Australian Government Medical Research Future Fund (2022:APP2018361).
Supplemental Material
Supplemental material for this article is available online
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
