Abstract
The discipline of human factors and ergonomics is largely focused on principled development of generalizable solutions. The process is typically slow, spanning months, even years. A crisis such as the COVID-19 pandemic presents a different problem. How can human factors and ergonomics professionals react quickly, within hours or days, to provide viable solutions to unanticipated problems as they become apparent? Here we report on a small project in which we were able to respond rapidly to an emergent COVID-19 requirement. Given time constraints, we had no opportunity to follow a systematic analysis and design strategy. Our development and testing strategies reveal lessons that can be applied more generally to development of human factors and ergonomics interventions within emerging crises.
Keywords
The human factors/ergonomics (HF/E) profession as a whole has offered to help with the coronavirus disease 2019 (COVID-19) crisis as it emerged in the early months of 2020. For example, Human Factors, the journal of the Human Factors and Ergonomics Society (HFES), issued a call for papers for a special issue consisting of short reviews on how the HF/E knowledge base can help with various issues arising from the COVID-19 crisis. HFES also hosts an online community, HF/E and COVID-19, dedicated to all matters related to HF/E and the coronavirus and has hosted a webinar titled Facing the Coronavirus (COVID-19): Human Factors Considerations.
However, there are special challenges to facilitating immediate, in-the-moment responses to emerging COVID-19 issues. The aforementioned examples largely involve HF/E professionals talking among themselves, and the discussion is primarily oriented to medium- and long-term interventions. A couple of the speakers in the webinar offered evocative and detailed accounts of emerging issues that represent areas for potential HF/E interventions but no specific advice on how HF/E professionals could bring their expertise and knowledge to bear in the moment. Indeed, ours is an analysis and design profession that is geared to the realization of medium- and long-term gains. In contrast, frontline responders must remain ready to intervene in the moment. They often face emerging challenges, some of which could be ameliorated by a timely HF/E intervention. Many HF/E professionals would like to help but cannot connect with those needing assistance when the latter are fully immersed in crisis response.
In this article, we share the experience of an immediate contribution to protection of health care workers as they treated COVID-19 patients. A rapidly assembled international team of HF/E and aviation professionals and a frontline responder adapted traditional HF/E practices to design, test, and validate a checklist. The team completed the task in 3 weeks, working off-hours in Australia and the United States to address this challenge. Through sharing this experience, our goal is to illustrate how this can be done and to identify necessary foundations for future successful efforts like this.
The Problem
Because of their frequent exposure to SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2, the virus that causes COVID-19), frontline responders are at high risk of contracting the disease themselves (Sheridan et al., 2020). Assistance was requested by an experienced anesthesiologist (AM). 1 The request was for assistance in developing a checklist for intubation and extubation of COVID-19 patients undergoing surgery for a health issue not necessarily related to COVID-19. Intubation and extubation are aerosol generating procedures that, in the case of COVID-19 positive patient, could expose staff to the SARS-CoV-2 virus (Australian and New Zealand Intensive Care Society, 2020). The checklist was for use in Melbourne Private Hospital, Victoria, Australia.
Anesthesiologists do not normally use a checklist but intubation and extubation of a COVID-19 positive patient required many new steps, some of which are inserted within familiar step sequences. Out of concern for the safety of his own airway management teams, AM sought to develop a checklist that would cover essential airway management and infection control procedures, from initial donning of personal protective equipment (PPE), through intubation, extubation, patient recovery, and doffing of PPE. He already had access to a number of checklists developed in other health care settings for these procedures, but he viewed them as inadequate from a HF/E perspective. Given the new hazards presented by the novel coronavirus, they failed to provide the necessary detail for full support of all critical steps involved in the modified procedure.
On Friday, April 3, 2020, AM contacted GL 2 and requested assistance with development of a checklist in anticipation of COVID-19 positive patients requiring surgery. With the possibility that COVID-19 patients could be scheduled for surgery as soon as April 6, the request carried with it a sense of extreme urgency. A checklist that was at least workable was required within 3 days. GL posted a request for assistance to the HF/E and COVID-19 online community through the HFES website on April 3, noting the urgency and the unpaid nature of the project. Five HF/E and aviation professionals (the coauthors of this article), across time zone differences of 13 to 17 hours, assembled online within 24 hours. They collaborated through the weekend to have a form of the checklist ready for Monday, April 6 (Figure 1).
Event timeline relating to emergence of COVID-19 in Australia and this checklist project (COVID-19 situation in Australia depicted above the timeline in black font; project events depicted below the timeline in blue font).
Design
Background
Anesthesiology teams, although experienced in airway management, generally lack experience with the infection control procedures that are essential for the management of a highly infectious virus such as SARS-CoV-2. The general procedures for treating the surgical theater as a hot zone (i.e., potentially infected) contained within and separated from a clean zone are understood. This primarily involved ensuring that staff and materials are subjected to infection control as they pass between hot and clean zones. However, the detailed steps and processes are not familiar to most surgical staff. Each treatment center in Melbourne was left to work out infection control details and processes for their own situation.
The required checklist was to guide the airway management team through intubation and extubation procedures that had been modified to minimize infection risk to the airway management team from COVID-19-infected patients and to ensure that the virus remained isolated in the surgical theater and not transported into clean hospital areas. The checklist was for the airway management team only. It would not cover infection control for other members of the surgical team.
Operational Concept
In the COVID-19 scenario, airway management is accomplished by a team of four; an anesthesiologist who performs the intubation, ventilation, and extubation (designated Lead Intubator), an anesthetic nurse (Airway Assistant), another anesthesiologist who monitors progress and assists (Medical Lead), and a nurse stationed outside the door of the theater to perform any essential activity external to the theater (Runner). The Medical Lead guides the team through the checklist. Those using the checklist will be health care professionals with varying levels of experience, but most will not have had experience in the use of PPE in infectious environments.
Design Considerations
In early development, we questioned whether the checklist should conform to a do-then-confirm or a read-and-do strategy (Burian et al., 2018; Clebone et al., 2017). Within commercial aviation, the do-then-confirm strategy is used with normal checklists, which are for well-practiced procedures such as before engine start, before takeoff, and approach. In use of a do-then-confirm checklist, pilots progress through well-practiced flows in real time without reference to the checklist. Then, before moving to the next critical phase, they review the checklist to ensure that the critical items were not omitted. A do-then-confirm checklist is typically short, does not necessarily include all actions in the procedure, and does not necessarily have the actions listed in the order of the procedure (Verdaasdonk et al, 2009).
The alternative read-and-do strategy is used with nonnormal checklists, which are for infrequently encountered system failures and emergencies. With a read-and-do strategy, the checklist guides progression through the process by having one pilot read each item aloud and the other pilot actioning the item. Read-and-do checklist are most suitable for long, sequence-sensitive processes, and for unfamiliar processes or those executed infrequently. They are prevalent in high-risk industries such as nuclear power.
Read-and-do checklists list all procedural steps and can, for execution of well-practiced procedural sequences, be viewed as overly pedantic. The requirement to pause at each item compromises development of a smooth, continuous decision–action flow and, for those experienced with the task or domain, will generally increase workload with little to no subsequent benefit. Competent, experienced workers typically find read-and-do checklists awkward. Nevertheless, the requirement for infection control with COVID-19 patients introduces many new steps and leads to modification of some existing steps, rendering this a nonnormal checklist.
For example, in checking the fit of the cuff for leaks, 3 the Airway Assistant would normally place an ear near the mouth of the patient to listen for a leak, which would increase the risk of contact with aerosolized SARS-CoV-2 as breathed out by the patient. The checklist warned against this strategy and specified an alternate method of checking for a cuff leak. Another departure from normal procedures was to place the laryngoscope in a sealed bag after use on a COVID-19 patient.
Largely because of these variations from normal practice, a read-and-do strategy was preferred to a do-then-confirm strategy. In execution of the checklist, the Medical Lead would enunciate the checklist item, and the designated team member would complete the item.
Checklist Development
AM developed a sequence of steps for each of the subtasks in the surgery procedure as the basis for the checklist. A checklist developed for a similar purpose by The Royal Melbourne Hospital in Victoria, Australia, served as a model (Figure 2). Other available checklists (see as online resources) were overly long or digressive, although they did serve as sources of ideas.
Checklist for surgery involving COVID-19 patients (provided by The Royal Melbourne Hospital; used as a model in this project).
Checklist design was constrained by the fact that airway management procedures for COVID-19 positive patients were not previously well established. Thus, the checklist needed to serve as a real-time training aid in addition to procedure guidance/recall.
Initial Design, Version 1
Version 1 of the checklist followed a sequence of procedural steps as developed by AM but was absent any explicit consideration of HF/E design principles. The resulting product was shared between team members who critiqued it with reference to their relevant HF/E expertise and their familiarity with use of checklists in aviation, health care, and process safety. The checklist was adjusted in response to the critiques and delivered on the morning of Monday, April 6.
Assessment and Redesign
Because no COVID-19 patients presented for surgery in the week of April 6, elective surgeries were postponed, so that time, staff, and theaters were available for simulation assessments of the checklist. Subsequently, the checklist went through eight cycles of assessment and redesign over the next 2 weeks.
Because access to the hospital was restricted due to the COVID-19 outbreak, AM was the only member of the design team who could be involved in the assessments. He acted as a participant-observer in all assessments. These were undertaken on an opportunistic basis during times when AM was not committed to his professional health care responsibilities and when a surgical theater and other health care professionals were available for a simulation. Assessment simulations were conducted with AM sometimes acting as Medical Lead and sometimes acting as Lead Intubator. The surgical team worked through the checklist sequence. AM monitored the process, noting where participants skipped steps or were uncertain about what to do and noting where the step sequence did not work well. AM recorded issues as they arose and also recorded comments from other participants. Following an assessment event, AM described the outcome to GL (by telephone 4 ), who corrected minor issues (e.g., typographical errors, changes in item order) and shared major issues with other members of the team (by email, Dropbox, and Zoom).
Lessons Learned and Final Design Features
Early assessments revealed that medical leads would inadvertently skip steps in the checklist. Several countermeasures were employed and tested over the first three design cycles: highlighting key tasks in red font, underlining key tasks, and numbering all tasks. However, none of these countermeasures worked. This issue was resolved with a requirement for each task to be checked off by the Medical Lead as completed. We hypothesize that the visual reminder of the task history mitigated Medical Leads’ tendency to revert to the more familiar and well-trained airway management procedures used for nonpositive COVID-19 patients.
There was ongoing discussion throughout the assessment cycles regarding the resolution of the COVID-19 steps: should they be few and detailed or many and succinct. We opted to initially test many and succinct as a strategy for isolating and correcting specific phrasing that caused errors in execution and expanding description where necessary. Figure 3 shows the approximately middle third of the intubation procedure for the first version (left) and the final version (right) of the checklist. The red font in the left panel was intended to alert the intubation team to COVID-19 specific items. That design feature did not serve its purpose and was discarded. The square bullets in the left panel were intended to identify a new item as a strategy for assuring that items were followed in sequence. That design feature also did not serve its purpose. The square bullets were replaced with numbers in the next version, which also did not serve the purpose. The right panel shows checkboxes that we used in the final version. Almost all items in this short sequence were expanded with more detail from the first to the final version based on confusions and uncertainties as reported by the assessment participants.
Changes (in language, font, and color) from Version 1 (left panel) to Version 8 (right panel) of the checklist for a section of the intubation procedure.
AM informally assessed different wordings during simulations, retaining those that facilitated usability. A continuing concern with phrasing and detail stimulated a systematic review of the wording of each item (undertaken independently by GL and checked interactively with AM) guided by a selection of principles taken from NAV Canada (2015) and guidelines offered by Degani and Wiener (1993). The adjustments resulting from these checks were relatively minor.
It became evident late in checklist development for donning and doffing of PPE that doffing is not merely the reverse of donning. During doffing, mishandling of equipment and inappropriate contact with skin can compromise infection control. This had been evident to AM but not to other members of the design team, who pushed for consistency in forms of expression for the two. This was recognized as an issue only in later design-assessment cycles when the difference between the two was made explicit to all. Subsequently, the checklist for doffing is more detailed than for donning, with treatment of each piece of equipment specified as a separate item. For donning, all pieces of equipment are identified in one item.
Many surgical staff who participated in the assessment simulations were doing similar work at other health care facilities. They contributed useful ideas that strengthened the airway management process. Although some surgical staff described the checklists they had observed in use at other institutions as vague and insufficiently prescriptive of critical steps, they were nevertheless able to abstract useful features from their experience with them.
In addition, the simulations revealed limitations of the COVID-19 airway management process, which were adjusted accordingly. The most substantial reworking of the process was in the equipment list, where different equipment layouts were trialed, and the item list expanded from 18 in the first version of the checklist to 23 in the final version with some original items being deleted. The final version of the checklist is shown in Figure 4.
Checklist for airway management of COVID-19 patients
With 23 items in the equipment list, equipment layout emerged as an issue. At first, the collation and organization of the items on the equipment trolley was arbitrary and would differ between simulations. The difficulty in locating items disrupted workflow and led to inadvertent contamination of equipment. It became clear that standardization of layout would simplify both preparation and execution phases. Consequently, a standardized layout, largely consistent with sequence of use, was established (Figure 5).
Final equipment layout.
It became apparent throughout simulations that anesthesiologists doing this for the first time were not familiar with the use of checklists of this sort. The medical lead would often paraphrase the item rather than enunciate it as required (e.g., for the item avoid bag mask ventilation, use low tidal volumes only if necessary, instead of repeating the challenge as in the checklist, the medical lead might say something like “Now ventilate but don’t use a bag mask and keep tidal volumes low if we have to use them”). Correction of this issue required a degree of familiarization with the process and with correct usage of checklists. In addition, it became evident that the read-and-do strategy of waiting for a challenge, then executing, and then confirming execution was foreign to experienced airway management teams who were accustomed to moving smoothly through the intubation procedures. Subsequently, the read-and-do requirement to respond verbally on item completion was discarded in favor of the medical lead confirming by observation.
Discussion
HF/E is generally oriented to medium- and long-term developments. In that respect, our discipline is not well oriented to resolving in-the-moment issues such as those that are emerging within the current COVID-19 crisis. How can HF/E professionals react quickly, within hours or days, to provide viable solutions to unanticipated problems as they become apparent? This project has highlighted some of the challenges. Given time constraints, we had no opportunity to follow a systematic analysis and design strategy. Instead, our geographically dispersed team, relying on 80 years of collective HF/E and aviation work experience, developed the first version of a checklist for a novel and evolving procedure in less than 72 hours, with no time for a literature review and without explicit reference to HF/E design principles.
Later in the project, we were able to critique our early versions of the checklist with reference to published HF/E design principles. The general thinking on a do-then-confirm versus read-and-do strategies was valuable but there was considerable inconsistency in how systematically we could apply specific principles and not all improved the design. Although seeming relevant, we could not follow some principles because of organizational and task constraints, while others (e.g., numbering, confirming item completion) led us in the wrong direction.
Organizational constraints and time pressure made it impossible to conduct a formal evaluation. We adapted by conducting informal evaluations which proved invaluable. The multiple iterations helped test hypotheses and illuminate unanticipated procedural issues (such as reorganization of the equipment tray) in a systematic manner.
Throughout the project, we relied on readily available software and internet capabilities. We created the first version of the checklist in MS Word but used MS Visio from thereon. We used Dropbox to share resources, to document and track checklist changes, and to assist team development of project documentation. Finally, we used Zoom for project meetings, and we exchanged ideas by email. On reflection, this project shows that the HF/E discipline has the capacity to make meaningful contributions in the rapid and reactionary development cycle of a high-stakes problem. However, several prerequisites that contributed to our success should be further cultivated in both HF/E practice and training:
Preexisting relationships between operators and managers in the industry and HF/E professionals seem critical for effective mobilization of HF/E resources in response to crises. It is often through personal connections that those needing help can know what to ask for and how to ask. In this case, a preexisting working relationship between one frontline health care worker with some experience in HF/E who saw the need (AM) and one HF/E professional who facilitated the response (GL) was critical.
Close professional relationships between HF/E practitioners are necessary for wider mobilization of diverse resources. Here, the role of our professional association (HFES) was critical. The online HFES community forum provided the means to solicit support of other HF/E professionals worldwide in a time of crisis, which enabled rapid formation of our team.
Knowledge in the head (Norman, 1989) of HF/E design principles is critical to development of immediate solutions to emerging problems. In crisis situations there is little time to trace knowledge in the world (e.g., HF/E textbooks and standards, or published literature) and no time for fundamental research.
When frontline operators are responding to an ongoing crisis, it becomes difficult to secure their assistance as subject matter experts for participatory design. Therefore, any domain expertise HF/E professionals have will be invaluable. In our case, we created one opportunity and had one break: AM recognized that HF/E could solve his problems, and he made time to seek support. Then the lull in COVID-19 cases in his hospital allowed his participation in the design and evaluation of the checklist.
Project Limitations
As noted above, the information we could access for checklist design was not consistently helpful, which made assessment critical. However, assessment was compromised by organizational constraints that forced us to rely on an informal assessment method. It is not clear how this could be resolved in preparation for intervening in future crises. Although Human Factors (the journal) has called for short reviews of the HF/E knowledge base to help with various issues arising from the COVID-19 crisis, and could do so for other types of emergent crises, research results and principles always have to be tuned to the specifics of a situation. Notably, the window to develop an intervention would, in some crises, shrink well below the meager 3 days initially available for our project.
The only potentially robust answer to this problem is to be prepared. Safety critical organizations typically seek to do that by development of crisis management structures, although HF/E rarely has the opportunity to influence the design of those structures. A pervasive characteristic of major incidents is neglect of obvious issues (Lintern & Kugler, 2017), many of which are human factors related. Organizational structures established to protect against runaway crises work well much of the time, but with a crisis as destructive as COVID-19, much of the time is not good enough. Furthermore, organizational issues are a constant theme in reporting on large-scale accidents. Problematically, our professional discipline has struggled to gain traction at an organizational scale. Whether we can ever establish a more expansive, organizational approach to safety design and risk management as envisioned by Rasmussen (1997) is debatable, but that is what we need to work toward if HF/E is to be relevant in any response to an emerging crisis.
Conclusion
This experience is just one of many arising as part of this pandemic (e.g., development of tools for contact tracing, redesign of personal protection, design of consistent and influential messaging) where insights from HF/E professionals would benefit frontline interventions. This particular instance highlights the value of establishing cross-disciplinary professional relationships in a time of calm, maximizing online communication tools in a time of need, and adapting rigid practices to enable agile development of time-critical solutions. These are significant issues that could be addressed by the HFES as well as within individual professional development of the members of our HF/E community.
Footnotes
Notes
.