Abstract
Importance
Nontechnical skills are crucial in delivering critical and urgent patient care. Through our simulation module, we gear residents and interprofessional personnel with the knowledge and skills necessary to tackle complex airway emergencies and limit human error.
Objective
Develop, implement, adapt, and evaluate a novel interdisciplinary and interprofessional crisis resource management (CRM) simulation module for the management of complex airways.
Design
Simulation-Based Quality Improvement Project.
Setting
McGill University’s Arnold and Blema Steinberg Medical Simulation Center and a variety of hospital environments at the McGill University Health Centre in Montreal, Quebec, Canada.
Participants
138 residents (otolaryngology, anesthesia, pediatric emergency medicine) and allied healthcare professionals (nurses and respiratory therapists) participated in 20 unique scenarios.
Intervention or Exposures
From 2012 to 2022, modules occurred from 4 to 6 half days per year, structured as 3 to 4 simulation scenarios, each followed by debriefing sessions.
Main Outcome Measures
Participants completed self-assessment forms evaluating module satisfaction, CRM skill development, and narrative commentary. Quantitative and qualitative data were obtained and analyzed.
Results
Participants reported a significant perceived increase (P < .05) in all nontechnical CRM skills. Participants without previous CRM training reached comparable levels in CRM skills to those with such training. Increasing involvement of allied healthcare professionals, formal debriefing focused on role clarity, and increasing complexity of scenarios are identified as key elements for stressing CRM skills and consolidating lessons learned.
Conclusions
This module is among the first of its kind in otolaryngology given its interprofessional, longitudinal, and evolving nature, while providing an opportunity for residents to develop nontechnical skills through simulation. Its interdisciplinary and interprofessional nature is a key element to its success.
Relevance
This module aims to translate into positive results in patient safety and patient outcomes in challenging airway management scenarios. Implementing modules as continued medical education may help maintain proficiency overtime.
Keywords
Key Messages
Simulation-based training is crucial in helping residents develop nontechnical skills, reduce human errors, and ultimately improve patient outcomes.
Purely-technical or single-specialty training programs and insufficient in allowing appropriate scenario simulation and debriefing around team dynamics.
Complexification of scenarios via the incorporation of interdisciplinarity and interprofessionalism is crucial to create further learning opportunities.
Introduction
Nontechnical skills are recognized for their critical role in ensuring patient safety, improving patient outcomes, and optimally managing crises.1 -5 Their importance is apparent in airway management, demonstrated by their explicit mention in care guidelines. 6 Nontechnical skills have been variously conceptualized, including concepts such as communication, planning and role allocation, leadership and followership, and mobilizing resources.6,7 This collection of skills has become the focus of the vast majority of crisis resource management (CRM) training.
CRM skills can be considered synonymous to nontechnical skills, defined as the formation of individual and group actions in typical, emergent clinical situations (ie, decision making, group dynamics, and teamwork). 7 CRM training focuses on developing CRM skills in high-acuity but low-frequency events, and simulation-based training has been important to allow safe practice of skills in emergency situations. 1 Integrating training nontechnical skills into the medical field began in the 1980s, when David Gaba and his colleagues integrated CRM principles from aviation into anesthesiology training.7 -10 CRM has since been incorporated into the training of other specialties and medical professions, often within the context of simulation-based education.11 -16
With the strengths of simulation-based education, the importance of CRM skills, and the range of high-acuity and low-frequency procedures, Otolaryngology—Head and Neck Surgery (OTL-HNS) is an area rich for educational development. More specifically, airway management is a foundational skill for OTL-HNS learners that includes technical and nontechnical skill components best practiced in the simulation setting. However, literature exploring nontechnical skill training in OTL-HNS is sparse, as evidenced by a 2020 systematic review identifying just 3 studies on this topic. 17
CRM skills allow for effective team functioning. In healthcare settings, teams are often interdisciplinary (ie, collaboration between physicians from different specialties) and interprofessional (ie, cooperation with allied healthcare professionals). Evidence suggests these teams are critical for delivering high-quality care in cases requiring the expertise of multiple specialists.18,19 Airway management is 1 example where high-quality care is dependent on multiple clinical team members. We identified only 2 studies exploring the role of interdisciplinarity simulation training in OTL-HNS.20,21 Participants in the first study were graduating medical students, not current OTL-HNS trainees. The other study mainly reported on technical skill development, minimally reporting on CRM skill development.20,21 Despite its importance, literature describing simulation-based approaches in CRM skill development for OTL-HNS trainees within interprofessional and interdisciplinary teams remains scarce. Therefore, we report on the development, implementation, evolution, and evaluation of a simulation-based interdisciplinary and interprofessional module for nontechnical skill development in complex airway emergencies.
Methods
Study Setting and Participants
This interdisciplinary complex airway CRM simulation training module is the final educational activity of a longitudinal simulation-based curriculum on airway management. Other modules cover airway basics, advanced airway, and advanced pediatric airway. Focus on interdisciplinarity and CRM skills are added as modules progress. 22 For OTL-HNS residents, the curriculum contains 4 modules spanning residency. Whereas technical and CRM skills are developed in earlier sections of the curriculum, this module is the first where non-scripted, non-OTL-HNS participants are included in scenario development and simulation. This aims to create high stress scenarios with greater case complexity and provide opportunities to practice shared decision making and adaptive leadership skills.
Our module took place in a variety of educational and clinical contexts, including McGill University’s Arnold and Blema Steinberg Medical Simulation Center and hospital environments including the operating room (OR), post-anesthesia care unit (PACU), intensive care unit (ICU), and emergency room (ER) at the McGill University Health Centre in Montreal, Quebec, Canada.
Resident trainees from 4 departments (OTL-HNS, anesthesiology, emergency medicine, intensive care) and nonphysician healthcare professionals participated in the module and its research component. Completion of the module and evaluation forms was the only inclusion criteria, with no further exclusion criteria applied. Refusal to complete the evaluation form did not preclude module participation. Participants were excused from clinical duties to facilitate participation.
Module Design and Learning Objectives
For OTL-HNS residents, this module is part of a larger curriculum that teaches effective and efficient management of clinically-relevant airway crisis situations. 23 OTL-HNS residents in junior training years participate in training modules focused on introducing technical skills and the theory behind CRM training. The module presented in this study is the final module within the larger curriculum and translates these skills to crisis scenarios, where most participating OTL-HNS residents are in their senior (PGY 4-5) training years and participated in 1 to 4 iterations of the course. Other residents and nonphysician healthcare professionals participate as part of their own curriculum or as an independent learning opportunity. Other participating residents were also in senior years of training (PGY4-5), whereas nonphysician healthcare professionals were all in active practice. This OTL-HNS curriculum was developed and implemented in 2012. Since then, it has been iteratively refined in response to continued quality monitoring, input from participants and experts from the participating specialties, as well as input from experts in medical and simulation-based education.
Module-specific learning objectives were divided into knowledge-based and CRM skills-based. Knowledge-based objectives included understanding the specific issues related to interprofessional team training, which were addressed in a 30-minute lecture to start the day. 23 Clinical management and CRM skills were the focus of over 20 simulation scenarios (Table 1) with high-fidelity manikins and group debriefing, of which 4 were selected for each module iteration. These complex airway scenarios allow participants to learn CRM skills and recognize the role of system-based issues in patient safety. The CRM framework includes the following nontechnical skills: “communicating effectively,” “establishing role clarity,” “mobilizing resources,” “using all available information,” “allocating attention wisely,” and “anticipating and planning.” 7
Comparison of Selected Past and Recent Cohort Scenarios.
Abbreviations: PACU, post-anesthesia care unit; ER, emergency room; OTL-HNS, Otolaryngology—Head and Neck Surgery; RT, respiratory therapist.
Scenario design was inspired by Salas et al’s 23 concepts underlying successful teamwork and D’amour et al’s 24 sentinel review on interprofessional collaboration. Interdisciplinarity and CRM skills were emphasized by the diversification of simulation settings, leadership transitions in complex clinical scenarios, and limitations in specialist’s availability during scenarios.
Module Development and Evolution
Through the 10-year lifespan of our curriculum, we increased early exposure of simulation-based education and CRM training in residency. This greater volume of earlier practice necessitated complexification of scenarios for more senior learners, leading to module evolution through increasing scenario complexity as requested by narrative feedback.
Scenarios were made more complex by modifying patient-related and systems-related factors (Images 1-2). Patient-related factors include the number of sites of injury or of medical issues, severity and evolution of injury or illness, failure of initial management plans, as well as patient comorbidities (eg, need for massive transfusion protocol, concurrent intoxication). System-related factors include scenario setting and resource limitations (eg, overnight resident responding to a PACU complication), the number of involved medical professionals, and specialty-specific knowledge in patient management, which stresses appropriate communication and leadership during the scenario (eg, OTL-HNS resident introduced halfway into the scenario). A comparison highlighting the shift in complexity between past and recent scenarios is illustrated in Tables 1 and 2.
Participants prepare to establish a surgical airway in a simulated scenario.
A large interdisciplinary team coordinates patient care in a simulated scenario.
High-Fidelity Scenarios Used for Interdisciplinary CRM Training in Residents.
Abbreviations: PACU, post-anesthesia care unit; ER, emergency room; OTL-HNS, Otolaryngology—Head and Neck Surgery; FB, foreign body; ICU, intensive care unit.
Data Collection
Data collection occurred between 2012 and 2022. Data from early-module iterations in 2012 through 2014 (defined as “past”) and more recent iterations in 2018 through 2022 (defined as “recent”) were included. Interim data are excluded due to unavailability. The distinction between groups was made due to later groups having had more experience with CRM training within the larger curriculum in general and being trained with more complex scenarios, thus prompting the question of whether self-assessment scores would vary between groups. The COVID pandemic did not impact the late group’s ability to participate in the curriculum.
Surveys filled after module completion included self-assessment of perceived CRM skill improvement, and participant feedback. Participants self-assessed their skills prior to, and following module participation. Response options used a 5-point Likert scale (1 = “strongly disagree” or “poor,” and 5 = “strongly agree,” “excellent” or “expert”). Items targeting participant feedback included realism of scenarios, relevance, quality of debriefing, feeling better equipped for future practice, and perceived usefulness. 5-point Likert scale items, binomial responses, and open-ended questions were used (Table 3).
Questions Regarding Overall Course Perception in Past and Recent Post-Module Surveys.
Participant surveys underwent minor modifications across implementation years due to modifications made in response to continuous participant feedback. Therefore, 2 versions of the self-assessment surveys are provided (Table 4), representing “past” and “recent” module iterations. Despite the stronger focus on interdisciplinarity in the later surveys, CRM skills are assessed in all survey versions and have been coded to that effect for analysis (Table 5).
Questions Regarding Participant Self-Assessment in Past and Recent Post-Module Surveys.
Abbreviations: MD, Medical doctor; RT, Respiratory therapist.
*Denotes questions only asked to a subset of participants based on their clinical role.
Key Points of The Crisis Resource Management (CRM) Framework and Associated Survey Questions.
Data Treatment and Analysis
Data were collected for each participant cohort. In order to highlight the sustained benefit, analysis focuses on 2 groups: “past” (2012-2014) and “recent” (2018-2022). Evaluation and self-assessment forms shifted across time, requiring consolidation and transformation to ensure data comparability across cohorts: Surveys assessing course feedback had transitioned from binomial responses to Likert scale responses. These data were converted entirely into binomial responses, transforming Likert scores “4” and “5” to a “Yes” (1) and Likert scores “1,” “2,” and “3” to a “No” (0) to allow data comparison. The very small number of responses scored a “3” were transformed to a “No” in order to be strict with course feedback assessment.
Quantitative data: Data generated by survey items using 5-point Likert scales and binomial responses were analyzed using descriptive statistics. Data regarding self- assessment scores are presented as mean ± SD where applicable. Findings were tested using t-tests where appropriate, as well as F-test, two one-sided-t-tests (TOST) equivalence test and analysis of variance when mentioned. Equivalence between groups utilized a hypothesized mean difference of 0.25, as differences in performance lesser than this value were deemed insignificant due to the data’s discrete nature.
Qualitative data: Data generated by open-ended survey items were analyzed using conventional content analyses, focusing on CRM key points, interdisciplinarity, interprofessionalism, and soft skills.
Results
Demographics
Between 2012 to 14 and 2018 to 22, 175 people participated in the module and 160 completed evaluation forms. The past cohort had 88 participating resident trainees, whereas the recent cohort consisted of 53 resident trainees and 19 nonphysician healthcare professionals [nurses and respiratory therapists (RTs)].
Available demographic data identified 71 male and 64 female participants. Most residents were PGY-4 and PGY-5 level (83%). Most participants reported having prior CRM training, with a similar proportion in both cohorts [85% (past) and 82% (recent)].
Overall CRM Skills
Given statistical equivalence between self-assessment scores of past and recent subgroups of all participating specialties (F-test 2-Sample for Variances and TOST equivalence test), as well as similarity of survey content and participant population across cohorts, cohorts were combined for analysis of module effectiveness.
There was a significant improvement between pre- and post-module self-assessment scores for all specialty groups (Figure 1; P < .05 for all), including residents in OTL-HNS (3.46 ± 0.68; 4.19 ± 0.62), anesthesia (3.51 ± 0.69 vs 4.03 ± 0.68), pediatric emergency medicine (PEM) (3.46 ± 0.62 vs 4.13 ± 0.57), and ICU (3.50 ± 0.61 vs 4.35 ± 0.67), as well as nurses (3.57 ± 0.69 vs 4.06 ± 0.47) and RTs (2.88 ± 0.59 vs 3.60 ± 0.67).
Overall CRM skills. CRM, Crisis resource management.
Individual CRM Skills
Analysis reveals similar improvements in individual CRM skills regardless of specialty. Participants showed improved self-assessment scores in all individual CRM skills (Figure 2; P < .05 for all). These include “Anticipate and Plan” (3.48 ± 0.61 vs 4.19 ± 0.66), “Allocate Attention Wisely” (3.59 ± 0.70 vs 4.20 ± 0.64), “Use All Available Information” (3.33 ± 0.67 vs 3.96 ± 0.67), “Establish Role Clarity” (3.37 ± 0.62 vs 3.99 ± 0.61), “Mobilize Resources” (3.33 ± 0.61 vs 4.01 ± 0.61), and “Communicate Effectively” (3.35 ± 0.73 vs 4.01 ± 0.66).
Individual CRM skills. CRM, Crisis resource management.
CRM Skills and Prior CRM Experience
Prior experience with CRM training correlated to higher pre-module self-reported scores. This difference was significant for items (all P < .05): “Anticipate and Plan” (3.31 ± 0.56 vs 3.59 ± 0.60), “Allocate Attention Wisely” (3.37 ± 0.76 vs 3.80 ± 0.68), “Use All Available Information” (2.79 ± 0.70 vs 3.49 ± 0.66), and “Establish Role Clarity” (3.10 ± 0.88 vs 3.55 ± 0.56).
Residents without prior CRM training showed a greater improvement than those with such training in the following CRM skills (P < .05 for all): “communicate effectively” (1.00 ± 0.91 vs 0.62 ± 0.69, P < .05), “establish role clarity” (1.07 ± 0.87 vs 0.46 ± 0.64, P < .05), and “use all available information” (1.07 ± 0.80 vs 0.55 ± 0.72, P < .05). Self-reported post-module scores for individual CRM skills were similar regardless of prior CRM experience (Figure 3).
Individual CRM skills and prior CRM experience. CRM, crisis resource management.
Module Evaluation—Narrative Responses
Participants’ comments were positive and provided concrete suggestions for improvement. Appreciation of debriefing sessions and of scenario realism was noted, as was the module’s pertinence in an increasingly-interdisciplinary and interprofessional healthcare setting. CRM elements mentioned most frequently in comments showed the largest self-reported improvement in the scale data: “establishing role clarity,” “communicating effectively,” “using all available information,” and “mobilizing resources.” Table 6 summarizes key areas of focus of participant comments with illustrative quotes from all professions.
Survey Data Highlighting Narrative Commentary in Participant Responses.
Module Evaluation—Participant Ratings
Overall, 86% (24/28) of participants felt that the simulation was realistic, 96% (149/156) felt better equipped to function in an interdisciplinary team during a crisis, 94% (139/148) felt that the debrief session was beneficial, 97% (117/121) felt that the module was relevant to their practice, 89% (75/84) found the module useful, and 98% (117/120) believed that there was value to course implementation. In summary, the module was well received by participants, suggesting a high-quality educational experience (Figure 4).
Module evaluation.
Discussion
Despite simulation training and interprofessional education gaining popularity in other specialties, most training modules occur in silos and do not allow repetitive exposure and habit-building.25,26 Our longitudinal curriculum is unique within OTL-HNS, supporting longitudinal skill development and reflecting the complexity of interdisciplinary scenarios. Interdisciplinarity and interprofessionalism in this simulation module were valued by participants, as was learning the priorities of other professionals and the skills within others’ armamentaria.
Comfort and proficiency in CRM are crucial for airway emergency management. Studies have determined that poor CRM skills are associated with 70% to 80% of errors in crisis situations.8,27 -30 This simulation-based module addresses the gap in CRM skills training in interdisciplinary and interprofessional settings. All participants develop CRM skills, and those without previous CRM experience reach similar self-reported levels of CRM skills to those with previous training. To our knowledge, this is the first study to report on the continued effect and evolution of a longitudinal, high-fidelity airway crisis simulation module focused on CRM skill development within such a diverse participant cohort. With regard to OTL-HNS participants, this module is further unique as it is part of a larger curriculum employing a graduated learning approach throughout residency. 22
Having been in place for over 10 years, the module’s design underwent constant educational quality improvement in response to participant and facilitator feedback. The most notable changes include increased interprofessional involvement with the inclusion of RTs and nurses and implementation of increasingly-complex simulation scenarios, allowing for further stress on the need for CRM skills. Scenarios were complexified through the modification of patient-related and systems-related factors, allowing for CRM skills to be stressed further and shaping the distinction between early and late cohorts.
This module is unique as it among the first to present a continuously evolving shared learning opportunity across a diverse specialty and professional cohort while focusing on the development of CRM skills. Dermody et al 21 included graduating medical students in scenarios aimed to teach trainees how to recognize and perform specific skills related to airway emergencies but did not include other allied healthcare professionals and lacked CRM skills evaluation. Residents targeted in our module and longitudinal curriculum were senior residents who graduated from earlier skill-based learning sessions and therefore had shifted focus to CRM skills. Volk et al 20 described their simulation training sessions for anesthesia trainees, OTL-HNS trainees, and OR nurses in the OR, but did not directly report on CRM skills, positioning our study as a unique contribution to the literature.
Participant feedback was very positive with strong points including useful debriefings, realism of simulated scenarios, the value of the perspective of others, the importance of closed-loop communication, and the importance of avoiding working in “silos.” These simulated experiences were described as true-to-life, genuine, and convincing, creating a smooth transition to real crises where teamwork is essential. These comments attest to the quality of the module in teaching CRM skills and many of Salas’ concepts underlying successful teamwork. 23 In addition to reporting improved skills in communication, cooperation, coordination, and team cognition, comments reflect how the module addressed the lack of clarity in team roles and responsibilities, an element that has been defined by Salas as the “biggest team killer.” 23
This study is limited given the dependence on self-reported data, and lack of objective performance data such as rates of success, efficiency scores, or errors. While beyond the scope of the current study, transferability of CRM skills to new contexts and over time represents an important area of future research.21,31 Furthermore, broad understanding of CRM frameworks is useful in assessing teamwork in high-acuity situations, and evaluation of teamwork through a standardized guide (ie, Salas et al 23 ) can better identify the impact of modules on teamwork dynamics in future studies.
Furthermore, the lack of objective methods to measure skill proficiency is a limiting factor in this study, and in the field of medical education as a whole. More accurate and objective methods of assessment than the Likert scale as well as methods to evaluate long-term skill retention would allow for more objective assessment of the curriculum.
Conclusion
Interdisciplinary and interprofessional simulation-based modules within a longitudinal curriculum provided benefits for CRM skill development in residents and nonphysician healthcare professionals. Ultimately, this module aims to translate into positive results in patient safety and patient outcomes in challenging airway management scenarios. In particular, increased interprofessional involvement in simulation scenarios and complexification over time through various factors has permitted stressing CRM skills, furthering the educational value to participants. Given this module has been in place for over 10 years with multiple adaptations, it continues to prove its worth and by responding to participant feedback.
Footnotes
Acknowledgements
We would first like to acknowledge Dr. Noura Al Youha for her significant contribution to the development of the module over its longitudinal course. Further, we acknowledge the contribution of faculty from the disciplines of OTL-HNS, anesthesia, and pediatric emergency medicine, as well as from the professions of nursing and respiratory therapy who were involved the module and program design and modification. Further, we acknowledge the contributions of the hundreds of participating residents and of the staff of the Arnold and Blema Steinberg Simulation Centre for facilitating simulation sessions.
Authors’ Contributions
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Ethical Approval and Informed Consent Statement
Consent to participate was obtained from all participants prior to participation in the simulation training module via the use of general consent forms, describing anonymization of data and outlining future use of questionnaire data for this project. This study was approved by the Institutional Review Board (IRB) of the McGill Faculty of Medicine and Health Sciences: IRB Study Number A03-E23-12B.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.