Using a Critical Safety Behavior Scoring Tool for Just-in-Time Training for N95 Respirator Use

Abstract

Background

Historically, health care workers (HCWs) have exhibited marginal adherence to proper N95 respirator use. During the COVID-19 pandemic, HCWs with little to no prior training on N95 respirator use are relying on N95s as their primary respiratory protection. There is a need for simple, effective, and easily implementable just-in-time training (JITT) interventions to improve N95 respirator-related safety behavior. This study investigated two JITT interventions.

Methods

A pilot experimental pretest posttest study design was used to evaluate two training interventions for N95 respirator donning/doffing performance at a Midwestern hospital system. HCW participants were randomly assigned to an intervention: one used a 4-minute instructional video alone, while the other used the same video but added a video reflection intervention (participant watched and scored a video of their own performance). All performances were scored using a 10-point Critical Safety Behavior Scoring Tool (CSBST).

Findings

Sixty-two HCWs participated (32 video alone, 30 video reflection). The two groups’ CSBST scores were not significantly different at pretest. Averaged participant scores on the CSBST improved immediately following both interventions. Scores were significantly higher on the posttest for the reflective practice intervention (p<.05). Years of experience and frequency of N95 respirator use did not predict pre or post scores.

Conclusions/Applications to Practice

We provide evidence to support the use of a time-efficient JITT intervention to improve HCW N95 respirator donning/doffing practices during the COVID-19 pandemic and beyond. Hospital safety professionals should consider this type of training for HCWs required to wear respiratory protection.

Background

Historically, health care workers (HCWs) have exhibited marginal adherence to proper N95 respirator use (Brosseau et al., 2015; National Institute for Occupational Safety and Health [NIOSH], 2015; Nichol et al., 2013). Respirators are frequently used incorrectly by HCWs and environmental services workers (Beam et al., 2020; Azizi et al., 2020; Mason & Friese, 2020) who may receive even less training (Savoia et al., 2020). In the midst of the COVID-19 pandemic, HCWs with little to no prior training on N95 respirator use have relied on N95s as their primary respiratory protection (Mason & Friese 2020; Occupational Safety and Health Administration, 2020). Respiratory protection is effective only when the proper respirator is selected and correctly used and maintained within its known limitations. Improper fit, use, donning, and doffing of a N95 respirator increases exposure and contamination risks. Proper wear and donning and doffing have become even more critical given the adoption of extended and limited reuse during the COVID-19 pandemic.

The critical importance of strict adherence to proper personal protective equipment (PPE) procedures, including those for N95 respirators, has been demonstrated during the COVID-19 pandemic (Kambhampati et al., 2020; Nguyen et al., 2020) and in previous outbreaks of special pathogens (Lietz et al., 2016; Uden et al., 2017; World Health Organization [WHO], 2015). In the early months of the COVID-19 pandemic, HCWs in the United Kingdom and United States had up to a 12-fold increased risk of being diagnosed with COVID-19 compared to the general public (Nguyen et al., 2020), and in the United States, HCWs accounted for around 6% of adults hospitalized with COVID-19, with nursing-related occupations comprising over one-third of those hospitalizations (Kambhampati et al., 2020). Likewise, HCWs have faced significantly higher risks of infection of other diseases compared to the general public, including Ebola virus disease (WHO, 2015), tuberculosis (Uden et al., 2017), and H1N1 (Lietz et al., 2016). Improper doffing of contaminated respiratory PPE (and potential redonning in times of reuse) is one of the greatest risks of HCW self-contamination (Chughtai et al., 2018).

The risks associated with inadequate N95 respirator training to an occupational cohort that has historically displayed marginal adherence to proper N95 respirator use have been amplified during the COVID-19 pandemic by the deviations from accepted respiratory standards, such as adoption of practices that enhance risks of contamination (e.g., extended use, reuse) and lapses in training (Petsonk & Harber, 2020). There is a need for simple, effective, and easily implementable just-in-time training (JITT) interventions to improve N95 respirator-related safety behavior during this pandemic and beyond. This study investigated JITT that would be practical to implement in a hospital setting or could be easily adapted to be provided outside of the hospital setting.

Methods

Two N95 respirator use training interventions were evaluated at a Midwestern academic hospital system between August 2018—April 2019 using an experimental pretest posttest study design. HCWs from both clinical and ancillary services (e.g., clinicians, environmental services, patient care techs) required to enter airborne isolation rooms as part of their job responsibilities were recruited for the study via email or through unit administrators. These connections were primarily through nursing managers, nursing professional development specialists, and environmental services administrators. Participants completed a demographic questionnaire to obtain data on age and gender; job role (open-ended); degrees and year awarded; years of experience in health care; frequency of N95 use (i.e., daily, weekly, monthly, rarely, or never); and fatigue level on a scale of 0 (no fatigue) to 10 (worst fatigue ever).

After completion of the demographic questionnaire, HCWs donned and doffed a N95 respirator while being video recorded in an empty hospital corridor and room as their baseline pretest performance. They were then randomized to one of two interventions based on a randomization schedule created prior to recruitment. One intervention used an instructional video alone, in which participants watched the first 4 ½ minutes of a CDC training video on N95 respirator donning and doffing and performance of a seal check (NIOSH, 2018). The other intervention used the same video, but added a video reflection intervention in which the participant reviewed and scored a video of their first performance using a 10-point Critical Safety Behavior Scoring Tool (CSBST). Following the assigned intervention, each participant was again asked to don and doff a N95 respirator which was recorded and served as their post-test performance.

The CSBST was specific to N95 donning and doffing and included 6 measures for donning and 4 for doffing. Hand hygiene accounted for 4 of the 10 points. CSBST donning elements for scoring included 1) hand hygiene before donning, 2) application of the respirator to the face and adjustment of the noseclip, 3) top strap placed at crown of the head, 4) bottom strap placed at the base of the neck, 5) user seal check of the respirator (both positive and negative pressure), and 6) hand hygiene after donning. Doffing elements for scoring included 1) hand hygiene prior to beginning the doffing process, 2) touching only the straps at removal, 3) gentle movement of the respirator from face to the waste receptacle, and 4) hand hygiene upon doffing completion.

The research team used the CSBST to score all performances of donning and doffing for comparison and evaluation; the participant’s self-score was not utilized by the research team in their scoring process. A detailed description of the longitudinal study data (1-month post-intervention), as well as the study setting, scoring process, and full CSBST have been previously published (Beam et al., 2020). This report focuses on all of the CSBST scores collected before and immediately after the intervention, including those that may not have returned for the 1-month follow up of the previous study. The study was granted Institutional Review Board Approval (#346-18-EX) by the University of Nebraska Medical Center.

Results

Sixty-two HCWs consented to study participation and were randomized to two groups (32 video alone, 30 video and reflection). The two groups’ scores were not significantly different at pretest (video alone averaged 3.4/10 and video reflection averaged 3.9/10 at pre-test). Averaged participant scores on the CSBST improved immediately following both interventions (Figure 1) from 3.4 to 4.9 for video alone and 3.9 to 8.2 for video reflection. Scores were significantly higher on the posttest for the reflective practice intervention (p<.05). Years worked in health care and frequency of N95 respirator use did not predict pre or post scores, nor did it predict level of increased improvement. Fatigue predicted postintervention scores (rho=.268, p=.037) and increases (i.e., higher fatigue was associated with higher scores at post-intervention; rho=.269, p=.036).

Figure 1.

Average scores of health care workers on the 10-point N95 donning and doffing Critical Safety Behavior Scoring Tool for both video alone and video reflection intervention groups (n = 62)

Figure 2 details performance of each CSBST step for both intervention groups at pre- and post-intervention, as scored by the research team. For all CSBST steps except performance of a seal check, the video reflection group had higher post-intervention scores than the video alone group. The largest differences in post-intervention scores between the two intervention groups were related to performance of hand hygiene steps, with few video alone intervention participants performing hand hygiene steps during donning or doffing pre-or post-intervention. Although both intervention groups scored poorly on performance of user seal check pre-intervention (3% for both groups), 72% and 53% of the video alone and video reflection participants, respectively, performed a user seal check post-intervention.

Figure 2.

Performance of each step of the N95 donning and doffing 10-point Critical Safety Behavior Scoring Tool (CSBST) for both intervention groups as pre- and post-intervention (n = 64)

Discussion

This pilot study has important implications for N95 respirator JITT strategies to prevent HCW occupational exposure. Video reflection may be one intervention that improves compliance with critical safety behaviors for JITT on N95 respirator use. It is practical and time-efficient (less than 10 minutes) and could easily be integrated into orientation for new staff and/or provided as a rapid refresher training both from home or at work. Although this study used three different camera angles for recording, the intervention could easily be adapted for ease of use; for example, performance of N95 donning and doffing could be captured using a phone or tablet to allow for a simpler video review process. These adaptations to the video process would make the training mobile, which would allow the training or re-training to be provided from any space, including outside of the hospital setting, while still allowing for remote observation and review by trainers. This also creates the potential for utilizing this training for the general public or other occupational groups for which N95 respirator use has become necessary during the pandemic.

Neither years in health care or frequency of N95 respirator use predicted CSBST scores. In many cases, particularly during epidemic/pandemic response, hospitals might task their experienced wearers as trainers for new staff. This study, however, highlights that experience with N95 respirator use might not translate to proper wear; as such, trainers should not be selected based solely on years of experience. As the pandemic endures, there is a real risk of HCW burnout and complacency to certain safety behaviors, including proper N95 donning, doffing, and use, that could compromise personal safety and heighten exposure risks. The JITT strategies evaluated in this study have the potential to be effective refresher trainings to counter the complacency while offering a consistent training tool that could be a better educational intervention than relying on experienced wearers as trainers alone.

Results of this study emphasize the importance of—and lack of adherence to—proper hand hygiene use during donning and doffing of respiratory protection. Hand hygiene is necessary before donning, after seal checking, before doffing, and after removing the N95 respirator. Comprising four of the ten CSBST steps, each of the hand hygiene steps were performed less than 50% of the time at baseline by both groups; even after the video education only intervention, almost none of the participants performed hand hygiene after donning or before doffing (the video did not demonstrate hand hygiene performance). These findings support previous studies of poor HCW adherence to hand hygiene practices and highlight the need for specific training on hand hygiene use as it relates to respiratory protection, supervision of compliance, and the importance of integrating hand hygiene steps into respiratory donning and doffing procedures (Gould et al., 2017; Musu et al., 2017; Sands & Aunger, 2020).

The use of a CSBST, even without incorporation into a training intervention, can be used by teams across the country to ensure adherence to safe N95 respirator behaviors during donning/doffing. The CSBST is simple, with objective measures that could be easily incorporated into or separately used as a checklist for HCWs entering and exiting high-risk areas (e.g., use by the buddy system). While the focus of this study was specific to N95 respirator use among HCWs, the recent interest and adoption of N95s and other filtering facepieces respirators (FFRs) by the general public during the COVID-19 pandemic provides another use case for exploring N95-related JITT interventions and CSBST use.

This pilot study provides other opportunities for future research directions. Previous studies have acknowledged the benefits of retraining for continuation of correct respirator use (Harber et al., 2014), but further research into the appropriate training frequency and type for competency retention is warranted. Our pilot study provides evidence that a short video paired with self-reflection can provide significant improvement both immediately following the intervention, as well as sustained improvement on N95 safety behaviors for at least one month (Beam et al., 2020). During this pandemic, the adoption of such an intervention, assigned periodically to each HCW providing care to patients with COVID-19, could significantly impact HCW safety and minimize exposure risks posed by complacency. More research is needed to determine how often the training should be repeated to sustain competency. This intervention, and further investigation of training frequency, could have implications for the training of other critical safety behaviors related to infection control and high-risk procedures.

There were a few limitations to this study. Participants were recruited from a single Midwestern hospital system and were primarily nurses or environmental services workers. Expansion of similar studies to include diversity in geographic locations, facility size, and occupational roles are needed to better generalize and understand baseline trends and impact of JITT interventions. The study was conducted prior to the COVID-19 pandemic and may not represent current HCW N95 respirator practices. This study does not reflect any reuse of the respirator. Future studies may need to evaluate a training on this specific skill and challenges workers are finding related to reuse.

Implications for Occupational Health Practice

The safe use of N95 respirators is critical for HCW safety, both during the care of patients with emerging infectious diseases and in more routine procedures or tasks with the risk for aerosol generation. The rise of SARS-CoV-2 variants of concern and recent Ebola virus disease outbreaks in two countries in Africa reinforce the need for HCWs to remain vigilant to workplace safety practices. We provide evidence to support the use of a time-efficient JITT intervention to improve HCW N95 respirator donning/doffing practices during the COVID-19 pandemic and beyond. We detail adherence of HCWs for each of the CSBST steps used to assess N95 donning/doffing procedures during this study to highlight critical safety-related steps with observed poor compliance. We hope the CSBST can be informative to and adopted by HCW teams as a simple checklist to augment the more robust donning/doffing procedures in place at institutions around the country.

Applying Research to Occupational Health Practice

This study explored two easily implementable just-in-time training interventions to improve N95 respirator-related safety behaviors among health care workers (HCWs). A simulation approach was used to evaluate two training interventions: one intervention used a 4-minute instructional video alone, while the other used the same video but added a video reflection intervention (total intervention time was less than 10 minutes). All 62 HCW participants were scored using a 10-point Critical Safety Behavior Scoring Tool (CSBST). Averaged participant scores on the CSBST improved immediately following both interventions, but scores were significantly higher on the post-test for the reflective practice intervention (p<.05). This study provides evidence to support the use of a time-efficient just-in-time training intervention to improve N95 respirator practices among HCWs during the COVID-19 pandemic and beyond. Findings also highlight critical safety-related steps with observed poor compliance (e.g., hand hygiene, seal check).