Abstract
N95 particulate respirator masks are currently recommended for all healthcare workers who care for patients with suspected or confirmed coronavirus disease (COVID-19) when performing aerosol-generating procedures. The protection provided by N95 particulate respirator masks is dependent on the filter’s efficiency and seal quality. In this prospective randomised crossover study, we conducted the user seal check and the quantitative fit test on two readily available duckbill models of N95 masks, the Halyard Fluidshield® N95 (Halyard, Alpharetta, GA, USA) and the BSN Medical ProShield® N-95 (BSN Medical, Mount Waverley, Victoria) particulate respirator masks. We recruited a total of 96 anaesthetic staff, of whom 26% were of South-East Asian ethnicity. We found that both types of masks provided reasonably high fit test pass rates among our participants and there was no significant difference between the two brands (77% for the Fluidshield and 65% for the ProShield, P = 0.916). Ninety-two percent of the participants could find at least one well-fitted mask among these two types of masks. We also demonstrated that the user seal check had low accuracy and low concordance (kappa coefficient of 0.16 for the Fluidshield and 0.08 for the ProShield) when compared to the quantitative fit test, and hence was not a reliable method to test seal quality.
Introduction
Since 11 March 2020, the world has been facing a global pandemic of coronavirus disease (COVID-19), caused by the novel coronavirus, severe acute respiratory disease coronavirus 2 (SARS-CoV-2). The number of cases is still escalating, at the time of writing, with some countries facing resurgences of the pandemic. Healthcare workers may be at particularly high risk of contracting the infection. 1 During the severe acute respiratory disease (SARS) pandemic in 2003, 21% of the global cases were healthcare workers. 2 Anaesthetic staff are especially vulnerable because of the frequent performance of aerosol-generating procedures. A recent study showed that one in ten healthcare workers who were involved in tracheal intubation of patients with suspected or confirmed COVID-19 subsequently reported a COVID-19 outcome, which was defined as either being diagnosed with new COVID-19 infection or required self-isolation or hospitalisation with new symptoms. 3 It is therefore vital for healthcare workers to wear appropriate personal protective equipment (PPE) to mitigate the risk of infection.
N95 particulate respirator masks are currently recommended for all healthcare workers who care for patients with suspected or confirmed COVID-19 when performing aerosol-generating procedures.4–6 They filter at least 95% of airborne particles but are not resistant to oil according to the US National Institute for Occupational Safety and Health (NIOSH) N95 classification. 7 The protection provided by N95 particulate respirator masks is dependent on the filter’s efficiency and seal quality. 8 Unfortunately, as supported by the study of Shaffer and Janssen, 9 there are large variations of seal quality, depending on the type of mask used, wearer’s ethnic background, and training level. The Centers for Disease Control and Prevention (CDC) advises that quantitative fit testing should be carried out before the use of N95 masks. 10 Quantitative fit testing involves a particle counter which measures leakage around the seal and provides users with an objective numerical value that describes the respirator fit. According to the Australian Standard AS/NZS 1715:2009, fit testing should be done annually. 11 Australian guidelines for the prevention and control of infection in healthcare recommend fit testing to be undertaken based on state/territory jurisdictional requirements together with risk assessment according to the healthcare setting. 11 However, South Australia is thus far the only state that has implemented a state-wide fit testing programme for all healthcare workers.
Fit testing can be expensive because it requires the use of specialised equipment by a trained operator. Each test takes approximately ten minutes to perform. Private companies charge about A$100 per person. 12 Moreover, it can be logistically challenging, especially during a pandemic. Therefore, many hospital staff around the world and also in Australia,13–16 including ours, predominantly rely on fit check, also known as the user seal check. 17 It is simple and quick, performed by the wearer each time a N95 mask is worn as per the manufacturers’ instructions. In order to pass a fit check, the mask should draw in slightly towards the face during inhalation, and fill up with air during exhalation. There should also be no air leakage around the edges of the mask.
In this study, we examined two types of N95 particulate respirator masks, namely the Halyard Fluidshield® N95 particulate respirator masks (Halyard, Alpharetta, GA, USA) and the BSN Medical ProShield® N-95 masks (BSN Medical, Mount Waverley, Victoria). Both are commonly available duckbill models, with very limited information available in published studies to show their fit test performance. We conducted both the user seal check and the quantitative fit test on our anaesthetic staff. We tested each individual on both of these two types of N95 particulate respirator masks. We compared the fit test results between these two types of masks. We also investigated our anaesthetic staff’s overall success rate of acquiring a well-fitted mask based on these two readily available types of N95 masks. Finally, we assessed the concordance of the user seal check when compared to the fit test result.
Materials and methods
This was a prospective randomised crossover study. The local ethics committee, Melbourne Health Human Research Ethics Committee, approved this study (QA 2020084), which was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12620000688987).
We recruited anaesthetic staff members (including anaesthetic consultants and trainees) at the Royal Melbourne Hospital. All participants were healthy volunteers. We excluded participants who had active respiratory symptoms, beards or moustaches. We performed a user seal check and quantitative fit testing on two types of N95 particulate respirator masks, namely Fluidshield particulate respirator masks (small or regular) and the ProShield masks (small or regular) on each individual. Verbal consent was obtained before inclusion to the study.
Before the commencement of the study, all participants received a formal PPE training session, which included N95 masks, donning/doffing technique and user seal check method. They also received instructions about the quantitative fit testing procedure from a trained operator. The study was performed in a dedicated room with steady room temperature and humidity.
The quantitative fit test involved the use of a PortaCount machine (PortaCount Pro+ 8038; TSI Incorporated, St Paul, MN, USA). The machine was calibrated at the beginning of each testing day, using a high-efficiency particulate air (HEPA) filter. A particle generator (model 8026 Particle Generator; TSI Incorporated, St Paul, MN, USA) was used to generate saline particles during the testing. This was to ensure an adequate ambient particle count throughout the testing. The N95 particulate respirator masks used in the study required the insertion of a probe which was necessary for connection to the PortaCount machine via plastic sample tubing. The probe was inserted by one of the study authors (BK) in a uniform location on the masks. The masks were prepared in bulk prior to the commencement of the testing day.
The order of the N95 mask brand to be examined was randomly allocated in blocks of ten. 18 The randomisation was stratified into two groups: (a) South-East Asian ethnicity; and (b) non-Asian ethnicity. This was to minimise any variation in fit test success rate due to differences in different facial shapes in different ethnic groups. 19 Demographic data, including age, sex, ethnicity (South-East Asian or non-Asian), height and weight, were recorded.
The participant donned the first randomised mask and adjusted its position in front of a mirror. In order to prevent leakage through the probe on the mask, the participant had to attach a sterilised PortaCount sample tubing onto the probe and block the distal sample port with one finger during the user seal check. The participant performed a user seal check as per manufacturer’s instructions. 17 The wearer passed the user seal check if he/she could not detect any leakage around the edges of the mask by themselves and would then proceed to the quantitative fit test with that mask. If the participant detected a leak around the mask, then he/she used a different size from the same brand. The wearer then repeated the user seal check and reported whether there was still leakage. The participant then chose the better-fitted mask of the two sizes for the subsequent quantitative fit test. Only the result of the better-fitted mask was recorded.
The fit test was performed according to the United States Occupational Safety and Health Administration protocols,
20
which consisted of four exercises: (a) Bending over at the hips and returning to upright repeatedly while taking two breaths during the bend-over; this lasted for 50 seconds; (b) reading a standardised text aloud for 30 seconds; (c) moving the head from side to side for 30 seconds; and (d) flexing and extending the neck for 30 seconds. The test was observed throughout by a trained operator and any breach of the protocol was addressed by recommencing the test. A maximum of two attempts were allowed for each test. The result of the better score was used. The PortaCount machine measures the respirator fit by comparing the concentration of the saline particles outside the respirator to the concentration of the saline particles that have leaked into the respirator. The fit factor was calculated for each exercise by dividing the concentration of the particles in ambient air outside the mask by that inside the mask. An overall fit factor score is calculated by the following equation:
An overall fit factor of greater than 100 was considered a pass.
After completing the fit test of the first randomised mask, the participant then performed the same user seal check and quantitative fit tests as described above with the second brand of the N95 particulate respirator mask. The user seal check result and the fit factor scores (individual score for each exercise and the overall score) for both brands of the N95 particulate respirator masks were recorded.
If neither of the two brands of the masks passed the fit testing, we provided a third mask option—3M Flat Fold Particulate Respirator (3M Mechanical or Industrial Engineering, St Paul, MN, USA) as a ‘rescue’. We also recorded the fit check and fit test result of the rescue mask.
The primary outcome was to compare the quantitative fit test pass rates between the two duckbill N95 particulate respirator masks. Secondary outcomes included the user seal check result, the individual fit factor score for each exercise and the overall fit factor score of each type of the mask tested.
Sample size calculation
There have been no studies evaluating the quantitative fit test pass rate of the Fluidshield and the ProShield particulate respirator masks. The study of Lam et al. showed that the quantitative fit pass rate among the three different tested N95 masks could vary from 0.45 to 0.7. 21 Therefore, if we assumed the fit test pass rate of one type of mask to be 45% and the other type to be 70% to show a clinically important difference, we would need at least 70 participants in each group to provide a power of 0.8. There were 101 anaesthetic staff members (including consultants and trainees) in our department at the time of the fit testing. Therefore, we elected to conduct the testing over four consecutive days and aimed to fit test all of our anaesthetic staff members.
Statistical analysis
Data were recorded using a standard spreadsheet (Excel; Microsoft, Redmond, WA, USA) and were analysed using Stata 13.0 (Statacorp, College Station, TX, USA). Descriptive statistics such as means, medians and percentages were used to present the demographic data, user seal check results and quantitative fit testing scores and pass rate of both types of N95 masks. An overall fit factor of greater than 100 was considered a pass. The chi-squared test was used to compare the fit test pass rate between the two types of masks. The paired t-test or Wilcoxon signed rank test was used to compare the individual and overall fit factor scores between the two types of N95 masks depending on the normality of the data distribution. A P-value value of less than 0.05 was considered statistically significant. The accuracy of the user seal check was scored against the overall fit test pass/fail result. The accuracy was expressed as sensitivity, specificity, positive predictive value, negative predictive value and overall accuracy. Kappa statistics were used to examine the agreement between fit test and user seal check results. The kappa value was interpreted as follows: less than 0.41 was poor, 0.41–0.60 was moderate and greater than 0.60 was good in consistency.
Results
Over the four consecutive days of fit testing (2–5 June 2020), we recruited a total of 96 participants, all of whom completed the user seal check and quantitative fit testing of both types of N95 particulate respirator masks. Two anaesthetic members could not attend the fit testing because of sickness, two were not rostered to work during the testing days and one was occupied with clinical work. Fifty participants (36 non-Asians) were randomly assigned to have the Fluidshield particulate respirator masks tested first and 46 (35 non-Asians) were randomly assigned to have the ProShield particulate respirator masks tested first. Table 1 shows the participants’ baseline characteristics.
Participants’ baseline characteristics and randomisation order.
Values are expressed as mean (standard deviation), number (percentage).
The fit test pass rate was 77% for the Fluidshield and 65% for the ProShield particulate respirator masks, P = 0.916. The overall and individual fit factor scores of both types of masks are shown in Table 2. Fifty percent of the participants passed the quantitative fit tests for both types of masks. Eight individuals failed both types of masks, giving an overall success rate of 92% (i.e. 92% of the participants were able to acquire at least one well-fitted mask from the readily available duckbill N95 particulate respirator masks at our institution). All of the eight participants who failed both types of masks managed to pass with the ‘rescue’ 3M Flat Fold Particulate Respirator, with the overall median fit factor score of 200, which was the highest possible achievable score. There was no significant association found between the fit test pass rate and age, gender, body mass index, ethnicity or randomisation order.
Comparison of quantitative fit test results between Halyard Fluidshield® N95 (Halyard, Alpharetta, GA, USA) and BSN Medical ProShield® N-95 (BSN Medical, Mount Waverley, Victoria) particulate respirator masks.
Values are expressed as number (percentage), median (interquartile range [range]).
The sensitivity and specificity of the user seal checks were similar between both types of masks (90.5% and 22.7%, respectively, for the Fluidshield particulate respirator masks; 80.6% and 26.5%, respectively, for the ProShield particulate respirator masks, Table 3). The kappa coefficient was 0.16 for the Fluidshield particulate respirator masks and 0.08 for the ProShield particulate respirator masks. The results indicated that there was poor agreement between the fit test and the user seal check results for both types of masks.
Comparison of results between user seal check and quantitative fit testing for Halyard Fluidshield® N95 (Halyard, Alpharetta, GA, USA) and BSN Medical ProShield® N-95 (BSN Medical, Mount Waverley, Victoria) particulate respirator masks.
PPV: positive predictive value; NPV: negative predictive value.
Values are expressed as number or percentage.
Discussion
In this study, we found that the two commonly available duckbill masks, the Fluidshield and the ProShield particulate respirator masks provided an above average (>50%) fit test pass rate among our participants and there was no significant difference between the two brands (77% for the Fluidshield and 65% for the ProShield particulate respirator masks, P = 0.916). Ninety-two percent of the participants could find at least one well-fitted mask among these two types of masks. We also demonstrated that the user seal check had low accuracy and concordance, and hence was not a reliable method to test seal quality compared to the quantitative fit test.
The fit test pass rates of our two chosen duckbill N95 models were relatively high among all the different types of N95 masks, as shown in other studies.8,9,21,22 However, one study by Lam et al. reported a prevalence of 65% of gross leak detected by the quantitative fit test in the pouch-type Kimberly-Clark N95 respirators (Kimberly-Clark, Irving, TX, USA), 21 which were similar to the two mask types used in this study. Our higher fit test pass rate could be due to the study population chosen and the study design. Our participants were all well-trained anaesthetic medical staff, with 26% being of South-East Asian descent, whereas the study population of Lam et al. 21 were all Asian nursing students. Unfortunately, our study was underpowered to show any significant association between fit test pass rate and ethnicity. Yu et al. and Lin and Chen both investigated the effect of ethnicity and facial structure on N95 mask fit and found that those of Asian ethnicity tend to have lower fit test pass scores, potentially because the masks were designed in Europe or North America.13,19 Moreover, our study was designed to allow the participants to change the size of the mask if a leak was detected during the user seal check. This enabled the participants to choose a better fitted mask for the subsequent fit testing, hence potentially improving the overall fit test result.
All of our anaesthetic staff members managed to find at least one well-fitted N95 particulate respirator mask by the end of the study. The eight participants who failed both types of the duckbill N95 particulate respirator masks managed to pass the fit test with the ‘rescue’ 3M Flat Fold Particulate Respirator, passing with the highest possible median overall fit factor score. This was consistent with many studies, which showed a higher fit test success rate with the three-fold 3M 1870 models8,13,19,21,22 when compared to the cup or valve N95 mask models.
Similar to other studies,14,21 we found that the user seal check method is unreliable in detecting seal leak, with high incorrect pass and failure rates for both types of the N95 particulate respirator masks. This potentially increases healthcare workers’ exposure to airborne pathogens. However, many hospital staff around the world still rely on this method to check for seal quality because fit testing is costly and time consuming.13–16 Each participant took about ten minutes to perform a user seal check and a quantitative fit test on each N95 mask in this study. During a pandemic, there are additional logistic issues to consider related to fit testing. With the global shortage of N95 masks, institutions would face the dilemma of whether to consume extra N95 masks for the fit testing or to preserve the masks for clinical use. Secondly, extra infection prevention precautions need to be taken in order to avoid cross infection; for example, maintaining physical distancing, vigilance about disinfection of any contact surface, and hand hygiene during the test.
Although it was a labour-intensive exercise, this study was extremely beneficial to the anaesthetic staff of our department. Not only did we identify the best fitted N95 particulate respirator mask for each participant, we also used this opportunity to provide further education regarding donning and doffing of N95 particulate respirator masks, to explore the optimal way to tighten the seal, and to show our support to our healthcare workers during the pandemic.
One of the limitations of this study is that we only tested two types of N95 particulate respirator masks. Therefore, we cannot extrapolate this study to other types of N95 particulate respirator masks. However, these two duckbill models are commonly available, inexpensive, compactly packaged and similar to surgical masks, making them intuitive for the staff to use. Another weakness of this study is that our cohort of participants was relatively young, within the normal body mass index, clean-shaven and mainly Caucasian. Thirdly, our power sample size calculation was based on a 25% absolute difference in the fit test pass rates between the two types of masks. Therefore, we could not reliably detect a difference of up to 25% even if a true difference existed. Finally, although a better fitted mask can reduce the penetration of ambient aerosol, 23 this study did not prove that there was improved healthcare safety.
In conclusion, over 90% of our staff, who were PPE trained and clean-shaven, passed the quantitative fit test with at least one of the two commonly available duckbill N95 models, the Fluidshield and the ProShield particulate respirator masks. Both types provided a relatively high fit test pass rate. We could not exclude (or confirm) a difference of up to 25% in the fit test pass rates. A fit test programme should be considered given the low accuracy of the user seal check technique, and bearing in mind the potential benefits of providing valuable support and education. However, this needs to be balanced with the availability of N95 particulate respirator masks during a global pandemic.
Footnotes
Data sharing
All the individual de-identified data that support the findings of this study are available on request from the corresponding author. Study protocol and statistical analysis are also available. Information will be available immediately following publication until five years after publication.
Acknowledgements
The author(s) would like to thank all the anaesthetic staff members who participated in this study, and Ms Yinwei Chen and Ms Teresa Sindoni for coordinating the study.
Author contribution(s)
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
