Abstract
BACKGROUND:
Neonatal Resuscitation is a required competency for pediatric and family medicine residency programs. Simulation-based training can be used to supplement clinical experience. Rapid Cycle Deliberate Practice (RCDP) has been validated as an effective education model and is gaining favor over traditional simulation models. The aim of this study was to evaluate the effectiveness of a simulation-based rapid cycle deliberate practice (RCDP) intervention on extremely low birth weight (ELBW) infant resuscitation.
METHODS:
Pediatric and family practice residents were randomized to control and intervention groups and participated in pre- and post-NICU rotation simulations. The intervention group received one RCDP session. Simulations were scored by blinded video review for overall performance, positive pressure ventilation (PPV), endotracheal intubation and behavioral skills. Surveys assessed confidence in ELBW resuscitation.
RESULTS:
Forty-one residents participated in the study. The RCDP group performed better than the control group at post-rotation evaluation for overall resuscitation performance (65% vs 87%, p = 0.004), administering PPV (63% vs 88%, p = 0.006), and validated behavior skills (1.4 vs 2.0, p = 0.019). Residents in the RCDP group reported greater confidence with ELBW resuscitation.
CONCLUSION:
An educational intervention using RCDP was associated with improved resident performance and confidence in ELBW resuscitation. RCDP should be considered for NRP and ELBW resuscitation training.
Introduction
Neonatal resuscitation is an essential competency for pediatric residents. Bag-mask ventilation, neonatal intubation, and umbilical line placement are required procedural competencies for pediatric residents per the American College of Graduate Medical Education (ACGME) [1]. In spite of this requirement, there is significant variation in the amount of hands-on training that residents receive during their training across different programs and centers. In many large training institutions, the resuscitation of ELBW infants is directed by neonatal fellows, or advanced practice practitioners, allowing for limited resident hands-on experience. According to a survey conducted among general pediatricians, 45% reported attending deliveries, and an additional 71% reported caring for sick newborns in the nursery [2]. Furthermore, 89% of practicing pediatricians reported residency mock code scenarios helpful in current practice [2]. Simulation training is a commonly used technique to supplement the clinical experiences of physicians in training. The Neonatal Resuscitation Program (NRP) enhances clinical experience with simulation and is foundational for the education and training of residents in newborn resuscitation [3].
The term “deliberate practice” was initially described by Ericsson as an intentional practice to achieve mastery of a skill [4]. The key to deliberate practice includes the ability to have guided practice in order to build necessary skills. Building on Ericsson’s work, Hunt described a competency-based educational model called “Rapid Cycle Deliberate Practice” (RCDP). In this model, learners alternate between active deliberate practice and expert-directed feedback until the skill or task is mastered [5]. During an RCDP session, a simulation is stopped at various time points for brief, targeted feedback. The simulation is then started again at the beginning. This cycle of stopping and restarting continues throughout the simulation, allowing for opportunities to master each of the individual skills being trained.
Peratta et al. noted that ideal scenarios for RCDP have some specific characteristics. These include scenarios with prescriptive performance guidelines, as often used in life-support courses. Other scenarios characteristic ideal for RCDP are high-risk, low-frequency events, including cardiac arrest or a code event. Lastly, those areas requiring complex team performance are well suited for RCDP [6]. Given this framework, delivery room resuscitation is an ideal scenario for RCDP. A recent scoping review by the International Liaison Committee on Resuscitation (ILCOR) noted that RCDP had been well described in other scenarios, but has minimal research in the context of neonatal resuscitation and warranted further study [7]. ELBW resuscitation is a complex process requiring many steps and team coordination. Given the extreme prematurity of infants, resuscitation more frequently requires procedural interventions. We conducted the following investigation to evaluate the effectiveness of simulation-based RCDP for resident education in ELBW resuscitation. We hypothesized that participation in RCDP training would improve resident skills and confidence with ELBW resuscitation.
Methods
Study design
This was a randomized control study conducted at University of Texas (UT) Health San Antonio and the affiliated hospital University Health System from July 2021 to June 2022. The study was deemed exempt by both institutions’ Institutional Review Boards. All participants completed written informed consent.
Participants
All first- and second-year residents rotating in the Neonatal Intensive Care Unit (NICU) were invited to participate in this study. This included residents from UT family practice, UT Pediatrics, and Brooke Army Medical Center (BAMC) Pediatrics residency programs. All participants had taken the NRP provider course at the start of their first year of residency.
Study design and randomization
All participants took part in a baseline (pre-test) simulation during the first week of their four-week NICU rotation and a final (post-test) simulation during the fourth week of the rotation. All participants completed a pre- and post- survey to assess demographics, experience and perceived confidence in ELBW resuscitation. Participants were randomized to receive standard NICU rotation education (control group) or the RCDP intervention in the second or third week of their rotation (RCDP group). Randomization occurred within each resident year and program (BAMC military pediatrics, UT family practice, and UT pediatrics) via random selection from opaque envelopes.
Simulations
The scenarios for the baseline and final simulations were similar, the resident led the resuscitation of a 25-week gestation ELBW delivered precipitously at a remote hospital without a NICU. In the scenario, the neonate presented with apnea and bradycardia requiring positive pressure ventilation (PPV), intubation, chest compressions, umbilical line placement and administration of epinephrine in order to establish a heart rate above 100. All scenarios were conducted using the Premature Anne simulator (Laerdal Medical, Stavanger, Norway) on a standard infant warmer which was appropriately stocked with neonatal resuscitation supplies.
The total scenario time was ten minutes, including a two-minute preparation period. Simulations were stopped at ten minutes regardless of infant status. A brief five-minute NRP algorithm-based debriefing session was conducted at the simulation’s completion. The simulations were performed by groups of three participants, the resident as the lead resuscitator and two embedded participants playing the roles of a respiratory therapist (RT) and a registered nurse (RN). Throughout the simulation, the RT and RN assumed the roles of inexperienced practitioners who recently completed the NRP provider course but had never participated in an actual ELBW resuscitation. The RN and RT actors were instructed to only follow instructions given by the resident leader and always ask clarifying questions like “What pressures would you like me to deliver the bag-mask-ventilation?” They were instructed not to offer suggestions or corrections.
Both the pre-test and post-test simulations were video recorded with a GoPro HERO 10. An independent reviewer evaluated and scored video recordings, blinded to participant group randomization.
Interventions
Participants randomized to the control group received standard bedside NICU teaching and participated in ELBW deliveries as they occurred throughout their NICU rotation. They received no additional dedicated education. Participants randomized to the RCDP group participated in a single 60-minute RCDP simulation. During this simulation, participants were paused at specific points, feedback was given, and they were asked to go back and repeat the process from an earlier point. For example, the simulation was paused after the initial preparation time, feedback was given, and then the simulation was restarted at the beginning of the scenario. Similarly, the simulation was paused after intubation, brief feedback was given, and then the simulation was restarted at the time of the infant’s birth.
In preparation for the RCDP simulation session, participants in the RCDP group also watched videos on PPV and endotracheal intubation (20 minutes total) and attended a NRP-based ten-minute didactic session on the principles of premature deliveries [8]. The total educational intervention time for the RCDP group was therefore 90 minutes (30 min didactics, plus 60 min simulation). The same researcher conducted all RCDP interventions to maintain consistency.
Outcomes
The primary outcome was neonatal resuscitation performance. Performance was evaluated using the Neonatal Resuscitation Performance Evaluation (NRPE). This scoring tool was chosen based on its prior validation as well as its ability to subdivide resuscitation scores into specific subdomains [9–11]. The total score for the NRPE is a sum of scores for 7 categories of NRP skills, 1) preparation and initial steps, 2) communication of heart rate, 3) bag/mask ventilation, 4) intubation, 5) chest compressions, 6) medication administration, 7) umbilical vessel catheterization. The total possible score for NRPE is 30; however, for this study, four items were non-applicable for the scenario, making the total possible score 26. Items removed included: 1) infant positioning on the warmer as infant was placed on the warmer by the facilitator, 2) removal of wet linens—as no wet linens were used, 3) free flow oxygen as this simulation necessitated forgoing free-flow oxygen and going directly to PPV, and 4) the size of equipment used was removed as it was difficult to see on video review. Lastly, an umbilical vessel catheterization (UVC) subdomain score was added and to achieve critical resuscitation elements. The full description of NRPE modification is described in a prior study [9].
Secondary outcomes included performance of positive pressure ventilation (PPV), endotracheal intubation (EI), and behavioral skills. PPV skill was evaluated using a previously validated checklist developed by the International Network for Simulation-based Pediatric Innovation, Research, and Education Research Network (INSPIRE) group [12]. EI skill was evaluated using a validated INSPIRE checklist for endotracheal intubations [13]. Behavioral skills important in neonatal resuscitation were evaluated using the Behavioral Assessment Tool (BAT), a rating tool developed to measure each of the ten key behavioral skills identified by NRP [8, 15]. Scores on the BAT have been shown to positively correlate with technical NRP skills [16]. Pre- and post-study multiple choice and free response questionnaires were used to capture perceived confidence with ELBW resuscitation. Examples of these questions included “I am confident as the team leader in routine NRP” and “I am confident as the team leader in NRP resuscitations of ELBW (<1000 gm) infants.” Confidence items were scored on a five-point Likert scale from “strongly disagree” to “strongly agree”.
Statistical analysis
The sample size calculation was based on a predefined educational goal to increase neonatal resuscitation performance by one standard deviation from baseline [9]. We calculated an a priori sample size of 18 residents per group would be needed to show a 10% increase in performance on the NRPE based on prior studies demonstrating an initial performance on the NRPE of 70% with a standard deviation (SD) of 10% [9]. Continuous data were summarized as median with interquartile ranges, while categorical data were expressed as a number with percent. Data comparison of scores was performed using T-test, Wilcoxon rank-sum test, and Pearson’s chi-square, where appropriate. Data were analyzed using R software version 4.0.1. A p-value of < 0.05 was considered statistically significant.
Results
Subject demographics
Forty-three participants were enrolled in the study and 41 completed the study over a period of 12 months (academic year 2021–2022). One participant withdrew prior to initiation of the study due to scheduling conflict, and another withdrew because of illness. In total, 22 were randomized to the RCDP group, and 19 randomized to the control group (Fig. 1).
Enrollment Flow Diagram.
Complete baseline demographic information is noted in Table 1. No differences were noted between groups. When considering the cohort as a whole, the majority (N = 27, 66%) of the participants were UT Pediatrics residents followed by BAMC Pediatrics (N = 9, 22%), and UT Family practice residents (N = 5, 12%). Twenty-seven participants were first year residents (66%). About half of the residents planned on pursuing fellowship training with 25% of those considering NICU as a possible fellowship. All had completed NRP with 78% having completed the course within the previous 12 months. While it did not reach statistical significance, there were more participants in the intervention group who had taken NRP < 3 months prior to the study (45% vs 21%, p = 0.5). All participants in both groups had attended deliveries but were relatively inexperienced as team leaders in routine NRP. Only six residents (14%) reported being a team leader for NRP in > 5 deliveries. Sixty-six percent had never attended a delivery of an ELBW infant, and none had led the resuscitation team for an ELBW infant. At baseline, no resident had successfully intubated a neonate (< 28 days).
Baseline demographic characteristics among the groups prior to study randomization
Univariate Analysis by Group Fisher’s exact test; Pearson’s Chi-squared test. 1. Brook Army Medical Center 2. University of Texas 3. Neonatal Intensive Care Unit 4. Extremely Low Birth Weight 5. Neonatal Resuscitation Program.
Baseline simulation scores were comparable between the groups in terms of objective measures (control vs intervention): NRPE (42% vs 44%, p = 0.5), PPV checklist (55% vs 44%, p = 0.5), EI checklist (20% vs 20%, p = 0.7), and BAT (1 vs 1.2, p = 0.4) (Fig. 2). Additionally, survey data collected showed similar baseline levels of confidence in both NRP and acting as a team leader. Table 1 details the remaining baseline measures.
Cow plot showing box and whisker plots describing pre- and post- intervention scores for both control and intervention groups Neonatal Resuscitation Performance Exam (NRPE), Positive Pressure Ventilation (PPV), Intubation, and Behavioral Assessment Tool (BAT). Boxes represent interquartile ranges. Solid lines inside boxes represent the median score of each group. Upper and lower whiskers represent the 75th and 25th percentiles, respectively. Single points above and below whiskers represent outliers.
In the final simulation, which occurred at the end of the four-week NICU rotation, objective performance of both groups improved from baseline in each of the four analyses (Fig. 2). There was an improvement in, the total modified NRPE score in both the control and intervention group. The overall score was significantly higher in the intervention group compared to the control group and intervention group (87% vs 65%, p = 0.004). The percentage correct for each section of the NRPE in the pre-test simulation and post-test simulation for both groups is shown in Table 2. The RCDP intervention group showed greater improvements in the mean post-test scores for the categories of chest compressions (60% vs 100%, p = 0.012), medication administration (66 vs 83%, p = 0.008), and UVC placement (33% vs 66%, p = 0.016). No differences were noted in baseline successful intubations between the two groups (21% vs 9%, p = 0.4), or in successful intubation in the post-test simulation (63% vs 82%, p = 0.2).
Neonatal resuscitation performance exam (NRPE) categorical and total scores between the groups
Neonatal resuscitation performance exam (NRPE) categorical and total scores between the groups
Median with Interquartile range. Bolded values for p value < 0.05. Analysis by Kruskal-Wallis test. 1. Neonatal Resuscitation Performance Evaluation.
There was a significant improvement in two of the secondary outcomes: PPV and BAT scores (Fig. 2). The PPV score in the control group was 63% compared to 88% in the intervention group (p = 0.006). The overall code behaviors as noted by the BAT score was significantly different between the control and intervention groups (1.4 vs 2.0, p = 0.019). There was no statistical difference in the EI checklist score between the control and intervention groups (33% vs 50%, p = 0.09).
The RCDP intervention group reported more confidence in acting as a team leader in deliveries for routine resuscitation (3.5 vs 5, p = 0.05), those requiring NICU admission (2 vs 3, p = 0.002), and ELBW resuscitation (2 vs 3, p < 0.001). The RCDP group also reported more confidence in performing chest compressions (2 vs 3, p < 0.001) and in the administration of epinephrine (2 vs 3, p = 0.029).
Discussion
We performed a randomized control trial to study the impact of RCDP to teach ELBW resuscitation skills to pediatric and family medicine residents. Our results suggest that RCDP is associated with improvements in ELBW resuscitation skills, including PPV delivery and resuscitation leadership behaviors. These results support the idea that the innovative RCDP teaching method enables rapid attainment of competence in resuscitation performance skills.
RCDP is an emerging simulation technique with direct applicability to neonatal resuscitation training. Our findings of improvement in resuscitation performance are consistent with prior studies. Hunt et al in their initial description of RCDP reported improvement in PPV, compressions and defibrillation in Pediatric Advanced Life Support education of residents [5]. Furthermore, RCDP methodology has demonstrated improvement in the treatment of septic shock simulations [17] as well as complications related to postoperative congenital heart defects [18]. Improvements in skill using RCDP are noted not only in residents but also in other healthcare professionals [18]. In addition to specific skills, Lemke et al showed improvement of overall team performance using RCDP [19].
With regards to RCDP in the context of NRP, limited data exist. In a study by Magee et al., using the megacode assessment form with pediatric interns, RCDP was compared to simulation with post-event debriefing. Results showed that RCDP interns showed improvement in immediate abilities as well as shortened time to perform critical tasks but did not show improvement in intern confidence or retention [20]. Furthermore, Patricia et al described a curriculum used for NRP with multidisciplinary teams. They describe the curriculum used as well as positive learner feedback and instructor education. They also concluded that NRP with its prescriptive algorithm provides excellent opportunities for RCDP [21]. Little has been published regarding ELBW resuscitation simulation training, and to our knowledge this is the first description of the use of RCDP to train residents to perform ELBW resuscitation.
Baseline scores in neonatal resuscitation skills for both groups were much lower than we anticipated. Overall NRPE scores were 43%, PPV scores were 50%, and EI scores were 20%. This is despite the fact that all participating residents had completed an NRP course. There are no current studies published to compare our data with others, but we speculate the reasons for the low baseline scores are likely multifactorial. Our cohort was notably inexperienced, with only 14% having been a team leader in more than five deliveries and only 34% having previously attended an ELBW birth. None reported successfully intubating a neonate. Pediatric residency is an ongoing immersive experience. Residents should gain delivery room and neonatal resuscitation experience during their NICU rotation. Participants in our study did receive modest delivery room exposure, and we did note some improvement in NRPE, PPV, and EI scores from baseline in the control group. However, the significant skill differences in the RCDP group as compared to control group on the post-test simulations suggests that our 90-minute RCDP educational intervention provides significant benefits over the routine NICU educational experience.
We found that our RCDP training intervention improved key neonatal resuscitation skills, but not all skills were better in the RCDP group. PPV is the most vital skill in neonatal resuscitation, as the vast majority of infants who need resuscitation will improve with appropriate PPV. PPV skills were significantly greater in the RCDP group vs control group as measured by the PPV checklist (Fig. 1). The bag valve mask score subsection of the NRPE did not reach statistical significance as it was only a four-question subset of a the larger NRPE (Table 2). EI scores were not significantly different on post-test evaluation (33% vs 50%, p = 0.09). One potential reason for this is the range of EI abilities of the participants. Resident EI scores during the post-test simulation ranged from 6 to 86%. This wide range of scores may be an artifact of our inability to spend significant time focusing on EI during the RCDP intervention and the lack of baseline intubation experience. Further, intubating ELBW is a difficult skill to master, and requires repetitive practice to achieve proficiency [22]. The post-test simulation intubation success in less than three attempts was 81% in RCDP group versus 63% in the control group. This suggests that the RDCP intervention had some educational impact on EI ability.
Behavioral skills important in neonatal resuscitation were significantly better in the RCDP group as compared to the control group, and residents who participated in RCDP had higher confidence levels. Neonatal resuscitation leadership is a difficult skill to master, but is crucial for pediatricians and family medicine residents, as they are often tasked to be the leaders in such events after training. Resuscitation of an ELBW infant is a complex process requiring many steps and team coordination. Successful resuscitation follows an algorithmic approach. Mastery of this algorithm is best achieved through repetitive, directed practice [4, 9]. We speculate the RCDP sessions enabled focused coaching, reinforcement, or immediate correction of the required behaviors and skills the residents needed to be successful. Our findings suggest that the short bursts of practice experienced in RCDP increase resident confidence. This increased confidence may have further improved technical and behavioral skills. One resident in the study commented, “I feel much more confident in both resuscitation skills and leadership in a resuscitation as a result of this experience.”
Our study adds to the growing body of RCDP literature and its use in resuscitation training. The results of our study showed improvement in technical skills, behavioral skills, and overall resuscitation performance. This suggests that RCDP is an effective training tool for ELBW resuscitation training for residents which could also be extrapolated to advanced practice providers and neonatal-perinatal medicine fellows. RCDP should be considered for ELBW resuscitation training and inclusion in future iterations of NRP.
Limitations
Our study has several limitations. This was a single-center study, which may limit the widespread applicability of the results. Additionally, only a few family medicine residents participated in the study. Another limitation is the short interval (four weeks) from pre- to post-simulation analysis. This interval was selected because of typical four-week NICU blocks completed at our program. Further, our study did not examine the long-term retention of skills and knowledge gained during the intervention. Lastly, while simulation is consistently used in educational environments, these skills do not always transfer to clinical settings. Further work should be done to determine if skills and behaviors acquired in RCDP simulation settings are transferable to actual neonatal resuscitation.
The strengths of this study include the use of a single RCDP facilitator with a small core group of actors, with standardized participant instructions prior to each simulation. Additionally, the use of a blinded reviewer using multiple validated tools allows for unbiased judgment of procedural skills and behavioral performance. This was augmented by the residents’ self-reported confidence reports.
Conclusion
We found that an educational curriculum using RCDP simulation was associated with improvements in ELBW resuscitation performance, behavioral skills, and confidence. The improvements shown here with RCDP are consistent with other studies from the pediatric simulation literature. RCDP should, therefore, be considered for training in ELBW resuscitations. Further work is needed to determine the long-term effects of RCDP, especially regarding retention of knowledge and skills, and the translation of skills gained in simulation to the clinical environment.
Footnotes
Acknowledgments
We thank the Pediatric and Family medicine residents at UT Health San Antonio and Brook Army Medical Center who participated in this study. We also thank the nurses and respiratory therapists who worked as volunteer actors for this study.
Funding
This study was funded by a Neonatal Resuscitation Program Human Factors Grant. This content is solely the responsibility of the authors and does not necessarily represent the official views of the NRP.
Conflicts of interest
The authors declare no pertinent conflicts of interest.