Abstract
Front-line staff routinely exposed to aerosol-generating procedures are at a particularly high risk of transmission of severe acute respiratory syndrome coronavirus 2. We aimed to assess the adequacy of respiratory protection provided by available N95/P2 masks to staff routinely exposed to aerosol-generating procedures. We performed a prospective audit of fit-testing results. A convenience sample of staff from the Department of Anaesthesia and Pain Medicine, who opted to undergo qualitative and/or quantitative fit-testing of N95/P2 masks was included. Fit-testing was performed following standard guidelines including a fit-check. We recorded the type and size of mask, pass or failure and duration of fit-testing. Staff completed a short questionnaire on previous N95/P2 mask training regarding confidence and knowledge gained through fit-testing. The first fit-pass rate using routinely available N95/P2 masks at this institution was only 47%. Fit-pass rates increased by testing different types and sizes of masks. Confidence ‘that the available mask will provide adequate fit’ was higher after fit-testing compared with before fit-testing; (median, interquartile range) five-point Likert-scale (4.0 (4.0–5.0) versus 3.0 (2.0–4.0); P<0.001). This audit highlights that without fit-testing over 50% of healthcare workers were using an N95/P2 mask that provided insufficient airborne protection. This high unnoticed prevalence of unfit masks among healthcare workers can create a potentially hazardous false sense of security. However, fit-testing of different masks not only improved airborne protection provided to healthcare workers but also increased their confidence around mask protection.
Keywords
Introduction
The coronavirus disease 2019 (COVID-19) pandemic has highlighted major deficiencies in global and Australian healthcare systems in the protection of healthcare workers from transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and other respiratory viruses. 1 The infection rate among healthcare workers varies between countries, and up to 29% has been reported. 2 Compared to similar countries, Australia seems to have relatively higher numbers of healthcare worker infections. 3 During the first wave in Victoria, 22% of healthcare workers who tested positive for SARS-CoV-2 acquired the virus through workplace transmission. 4 However, this increased to at least 69% during the second wave, 4 highlighting the urgent need for better protection for front-line healthcare workers.
Initially, the main route of transmission of SARS-CoV-2 was thought to be via droplets. 5 However, increasing evidence exists that a significant proportion of COVID-19 transmission is airborne via aerosolised particles. 5 A systematic review reported that staff exposed to aerosol-generating procedures (e.g. intubation) were at a 6.6 times increased risk of airborne infections during the SARS epidemic. 6 Airborne personal protective equipment (PPE) is recommended to include at least a N95/P2 mask when treating patients with COVID-19 and likely to be exposed to aerosol-generating procedures.7–9 A standardised annual fit-test is required to ensure an N95/P2 mask provides a good seal with the face of the healthcare worker, as set out by the Australian standard (AS/NZS 1715:2009). 10 Regular fit-testing is routine practice in US, Canadian and British hospitals, but is not routinely performed throughout Australia. 7
At the start of the COVID-19 pandemic, in the absence of a statewide fit-testing programme, a significant number of healthcare workers across many departments of different tertiary hospitals in Western Australia independently organised their own fit-testing and attended either a private fit-testing centre or underwent self-organised departmental qualitative fit-testing. This audit presents data of members of the Department of Anaesthesia and Pain Management of our hospital, who had undergone qualitative and/or quantitative fit-testing with the standard available N95/P2 masks at this institution during that time.
Methods
We collected the data from a convenience sample of staff who opted to undergo privately organised qualitative and/or quantitative fit-testing between 22 April and 11 August 2020. Data were prospectively collected as part of an approved institutional audit (approval: GEKO35136). All staff members consented by filling out the voluntary audit form for their data to be used and published. Independent of this audit, at the beginning of the COVID-19 pandemic, all staff members had a one-off departmental PPE training session including fit-checking. All staff members were provided with additional on-the-day PPE training including how to perform a fit-check prior to fit-testing and were asked to perform a fit-check before fit-testing. A fit-check is a process whereby a person ensures their own correct wear and good facial seal. 1 The order of qualitative or quantitative fit-testing was based on the availability of testing times, testing staff and the staff member’s personal schedule.
Masks tested
Up to and at the time of testing, only one N95/P2 mask, the Proshield® N95 mask (TN01; BSN Medical, Mulgrave, Victoria) in two sizes (medium TN01-11 and small TN01-12) was available to staff as airborne precaution PPE. The initial size of the N95/P2 mask used for fit-testing was the one deemed most appropriate at the time of the PPE training session and routinely used by the staff member when applying airborne precautions.
For this audit, if a staff member did not pass a fit-check, then the mask was readjusted especially around the bridge of the nose, a different mask size was tested, or the mask was taped and a fit-check was repeated prior to fit-testing. If the staff member did not pass a fit-test, then a different mask size was used or the mask was taped in an attempt to improve the seal. While taping is not recommended in the healthcare setting, at the time of this audit, taping was gaining popularity among staff members when a N95/P2 mask demonstrated a leak, as no alternative respirators were available. Some staff members were additionally tested with self-sourced masks. 3M™ N95 mask (9322A+P2; North Ryde, New South Wales) was the mask most frequently self-sourced. There was no limit on the number of fit-tests for an individual staff member.
Qualitative and quantitative testing were performed following fit-testing guidelines including predefined exercises. 11 International, national and local guidelines regarding social distancing hygiene (including cleaning of qualitative hood) were followed. 12 Some staff members underwent standardised qualitative fit-testing at another local hospital within the same period and offered their data for analysis.
Qualitative fit-test
A staff member trained in performing qualitative fit-testing performed the tests in a standardised manner using a commercially available kit (3M Qualitative Fit Test Kits, North Ryde, New South Wales). A bitter (Bitrex, Edinburgh, Scotland) or sweet (sucrose if asthmatic or failed bitter) taste was used to assess adequate fit. After the testee confirmed a bitter/sweet taste without a mask (sensitivity test), the testee with a mask indicated absence (pass, no leak) or presence (fail, leak) of a bitter or sweet taste.
Quantitative fit-test
Staff members, who voluntarily attended a professional quantitative fit-testing centre (Occuhealth, Mandurah, Western Australia), provided their fit-testing results for analysis. Quantitative fit-testing was performed using a PortaCount® (TSI 8038; Shoreview, MN, USA). An overall fit factor of 100 or greater was required to indicate sufficient fit (no leak). 11
Data captured
All staff members who attended fit-testing completed a short questionnaire. This included characteristics, prior N95/P2 mask training and the statements: ‘I am confident that my mask provides an adequate fit’ and ‘fit testing has improved my knowledge on how to wear my N95/P2 mask safely’ measured using a five-point Likert scale: 1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; and 5 = strongly agree. For staff members who underwent both qualitative and quantitative fit-testing, answers are reported on the first method of fit-testing if not otherwise specified. Fit-check data prior to internal qualitative fit-test were prospectively recorded. We documented mask type and size, whether the mask was taped, pass or failure and duration of fit-testing. Each qualitative test was timed individually, whereas the total duration for all quantitative fit-tests was measured.
Data analysis
First fit-pass rates can be seen as a measure of healthcare worker protection when a healthcare service does not include regular fit-testing in their respiratory protection programme. Final, as compared to first, fit-pass rates provide an estimation of how many healthcare workers can obtain adequate airborne protection given sufficient testing of different masks.
We used results on first tested mask and first mask without leak to represent first and final fit-pass rates, respectively. To analyse both initial and final fit-pass rates we used the first attended fit-test method (qualitative or quantitative) and excluded fit-test results of taped N95/P2 masks, as taping N95/P2 masks is not recommended in healthcare due to the increased risk of viral transmission during doffing. 13 In addition, qualitative and quantitative fit-pass rates were compared (untaped and taped masks) when a staff member tested the same mask with both fit-test modalities.
SPSS (Statistics 26.0; IBM, Armonk, NY, USA) was used for statistical analysis. Data are presented as numbers of total cohort (percentage) or median (interquartile range) if not stated otherwise. Data were analysed using chi-square, kappa coefficient, Wilcoxon signed-rank or Mann–Whitney U-test as appropriate. A kappa value greater than 0.70 was taken to support agreement between two tests. 14 A P-value of less than 0.05 was considered significant.
Results
This audit presents fit-test results of 53 doctors (predominantly anaesthetists) and 31 non-doctors (predominantly anaesthetic technicians). Staff member characteristics are shown in Table 1. All staff members completed a qualitative fit-test, 70 (84%) within the department and 14/84 (16%) elsewhere. Quantitative fit-testing was completed by 28 (64%) before and 16 (36%) after the qualitative fit-test. Only seven out of 84 (8%) staff members had formal N95/P2 mask training before COVID-19, whereas 69/84 (82%) had their first formal training during the emergence of the pandemic; the remainder only received training prior to the fit-testing. Only 59/84 (70%) reported being familiar with fit-checking. Only one staff member had been fit-tested, overseas, before COVID-19. All staff members were clean-shaven.
Characteristics of staff members undergoing fit-testing.
Data are given as n (%) or median (interquartile range).
Ethnicity as identified by the healthcare worker.
Clinical experience since graduation.
First and final fit-pass rates without tape for all participants are detailed in Table 2.
First and final fit-pass rates in order of tested masks.
First attended fit-testing method (qualitative or quantitative) was used to determine the first and final fit-pass rate (no leak).
Order of tested masks mostly first Proshield® N95 masks (TN01; BSN medical, Mulgrave, Victoria) and then 3M® N95 masks (9322A+P2; North Ryde, New South Wales) if available.
We only show ‘untaped’ fit-pass data.
First fit-pass represents protection provided by an institution to staff members using available masks without a fit-testing programme.
Percentage indicates the fit-pass rate.
Confidence ‘that the available mask will provide adequate fit’ (median, IQR), measured on a five-point Likert scale was higher after fit-testing compared with before (4.0 (4.0–5.0) versus 3.0 (2.0–4.0); P <0.001) (see Figure 1). Before fit-testing, 35/84 (42%) staff members agreed or strongly agreed that they were confident their respirator would provide adequate fit. After fit-testing, this increased to 73/84 (87%). Following the first fit-testing, confidence in protection adequacy improved in 57/84 (68%), decreased in five out of 84 (6%) and remained the same in 22/84 (26%). The majority 78/84 (93%) agreed or strongly agreed ‘that fit testing improved their knowledge on how to wear their N95/P2 mask safely’. Confidence in the adequacy of mask fit following fit-testing was associated with the staff member finding a fitted mask (no leak) (P<0.001). We could not find associations with other factors; for example, being Caucasian (P=0.979), a doctor (P=0.622), if tape was used to obtain the final fitting mask (P=0.730), or whether the fit-test was quantitative or qualitative (P=0.769).
Frequency distribution of Likert scale responses. Staff members were asked ‘I am confident that my mask provides an adequate fit’ before (hatched) and after (solid) fit-testing. Five-point Likert scale: 1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; and 5 = strongly agree.
No staff member failed fit-checks on all masks. Fit-check results were documented for 78/84 (93%). We found 41 (53%) false negative (leak when tested but not when checked) and 36 (46%) true negative (no leak when checked or tested). Therefore, the negative predictive value was 47% for an absent leak on fit-check to pass (no leak) a qualitative fit-test. As fit-testing was not performed if the fit-check failed, we were not able to estimate the true and false positive rate. The kappa coefficient between fit-check and fit-test results was –0.03.
Forty-four staff members attended both quantitative and qualitative fit-testing. For these, 74 paired mask fit-tests are available for comparison (e.g. untaped mask ‘A’ qualitative versus untaped mask ‘A’ quantitative). For 49 (66%) of these paired mask fit-tests, the quantitative fit-test was performed prior to the qualitative fit-test (Table 3). In this direct comparison for the same staff member, quantitative fit-testing had a higher fit-pass rate compared to qualitative, 74% versus 59%, P=0.006. There were 38/74 (51%) true positive events (qualitative and quantitative pass), six out of 74 (8%) false positive (qualitative pass, quantitative fail), 17/74 (23%) false negative (qualitative fail, quantitative pass) and 13/74 (18%) true negative (qualitative and quantitative fail). Therefore, sensitivity and specificity for the qualitative fit-test correctly to indicate pass (no leak) when compared with the quantitative fit-test method was 69% and 68%. The kappa coefficient between qualitative and quantitative fit-pass rates was 0.32. The average duration per fit-test was similar between quantitative and qualitative fit-test modality, 11 (9–14) versus 12 (10–15) minutes, respectively (P=0.780). The time to complete the first qualitative fit-test was longer than it was to complete the second test 13 (11–16) versus 9 (8–10) minutes, respectively, P<0.001.
Comparison of qualitative and quantitative fit-pass rates.
Paired data (n=74) were analysed of 44 staff members who attended both fit-testing modalities.
Proshield® N95 masks were TN01 (BSN Medical, Mulgrave, Victoria) and 3M® N95 masks were 9322A+P2 (North Ryde, New South Wales).
Discussion
At the start of the COVID-19 pandemic, fit-testing was neither offered nor recommended by the local health authorities, although fit-testing is an essential part of the national guidelines in order to comply with the Australian standard (AS/NZS 1715:2009).10,15 PPE, including N95/P2 masks, was in short supply; fit-testing capacity across the statewide health system of Western Australia was limited. 15 Concerned healthcare workers at different departments at various tertiary hospitals across Western Australia independently organised fit-testing, highlighting an imbalance between healthcare workers’ expectations and local institutions’ actions in regard to staff safety.
In this audit, the first fit-pass rate of the untaped N95/P2 masks (in line with routine use) was only 47% with the masks available at our institution. This unexpected low fit-pass rate may be specific to the standard mask type available in multiple large teaching hospitals. We cannot exclude a higher first fit-pass rate with a different N95/P2 mask. Of note, the types and sizes of masks provided to our staff during the pandemic were equal to those available before the pandemic. Importantly, without an appropriate fit-test programme, the concerningly high rate of healthcare workers with insufficient airborne protection will go unnoticed. This false sense of security poses a risk not only to the individual healthcare worker performing aerosol-generating procedures, but can also endanger other healthcare workers or patients, especially during a pandemic.
We found, by changing sizes of the available masks and offering a limited variety of alternative masks, an appropriate fit was eventually achieved in 79% of staff. This is in line with the literature, and higher final fit-pass rates have been provided when offering a larger range of shapes and sizes of N95/P2 masks.16,17 Hospital protection programmes are advised to make a variety of different types and sizes of N95/P2 masks available to provide for the different facial characteristics of healthcare workers. Health service providers should not primarily source N95/P2 masks from a cost perspective but with the aim of the best possible protection (fit) for healthcare workers. Asian or female staff are known to have lower reported first and final fit-pass rates,18,19 and will therefore greatly benefit from an appropriate mask selection. Elastomer half-mask respirators (EHMRs) or powered air purifying respirators (PAPRs) should be considered for healthcare workers who fail fit-testing. Alternatively, healthcare workers without a properly fitting mask may have to be moved to hospital areas not caring for potentially infectious patients. 20
Taping to improve seal is not recommended in clinical practice due to potential loosening of the tape and consequent loss of user seal as well as a higher risk of self-infection during the doffing of the taped mask. 13 The fact that healthcare workers were using non-recommended measures such as mask taping demonstrates increased angst around inadequate airborne protection provided by available N95/P2 masks.
Like others, 16 we found fit-testing to increase the level of confidence that the available mask will provide an adequate respiratory protection. That fit-testing improved confidence is not at all surprising, given that fit-checking is more accurate than fit-testing. 1 Whether the improved confidence of health workers regarding airborne mask protection also reduces angst during a pandemic has not been investigated.
In line with the literature, the negative predictive value for fit-checking to detect a leak was less than 50%, meaning that approximately half of healthcare workers are left with a false sense of protection.1,19,21,22 Many institutions, including our own, promoted a fit-check as a minimum standard when using N95/P2 masks without offering fit-testing.7,15 While it is imperative to check for seal before routine use of an N95/P2 mask, our data underline that fit-checking alone cannot replace fit-testing to ensure optimal mask selection.
The sensitivity of the qualitative fit-test to indicate pass (no leak) correctly was less than 70% when compared with quantitative fit-testing, although even lower sensitivities of (48%–58%) have been reported. 1 Despite the tests being performed using the same testing and exercise protocol, 11 we found a higher fit-pass rate for quantitative fit-testing which can be attributed to the subjective nature of the qualitative fit-testing 1 and increased anxiety during the pandemic. 23 We cannot exclude that the order of fit-test (e.g. quantitative before qualitative) influenced some of our results. However, previous studies have found quantitative fit-testing to be superior in detecting leaks.14,24 Overall, this audit supports quantitative over qualitative fit-testing because of the higher leak detection accuracy.
This audit has several limitations. It represents a small convenience sample of front-line healthcare workers during the COVID-19 pandemic. Due to the limited availability at the time, only a minimal selection of N95/P2 masks was tested. The order of the masks was not randomised and the method or order of fit-testing was not controlled. Furthermore, staff were allowed to use tape, contrary to standard recommendations. Ultimately, the results are specific to the masks tested, precluding the generalisability to other masks.
Nevertheless, this audit highlights that without a fit-testing programme, the number of healthcare workers with insufficiently protective N95/P2 masks is high and goes unnoticed. Wearing non–fit-tested N95/P2 masks can create a false sense of security. While the cost of incorporating fit-testing into a hospital respiratory protection programme may appear high, it is on a par with other in-person mandatory training and is relatively low when compared with potential sick leave or legal costs. 7
This audit clearly demonstrates that fit-testing is essential to guide the choice of correct fitting masks for appropriate protection. It also highlights that fit-testing increases knowledge around mask protection and staff confidence that their N95/P2 mask will provide adequate protection.
The Western Australian Department of Health is currently reviewing the establishment and implementation of a respiratory protection programme that includes a quantitative fit-testing component for some high-risk healthcare workers. 25 Since the audit data were collected, the hospital has sourced one size of an additional brand N95/P2 mask for those healthcare workers who detect a leak on fit-check.
Conclusions
Less than 50% of staff routinely exposed to aerosol-generating procedures were appropriately protected from aerosolised particles with the N95/P2 masks available at a tertiary Australian institution. In line with the literature, without a fit-testing programme the insufficient airborne protection of healthcare workers goes unnoticed, and wearing a non-fit N95/P2 mask can create a false sense of security. In contrast, implementing fit-testing improves confidence among healthcare workers that their N95/P2 mask provides adequate protection, and can also identify healthcare workers that are insufficiently protected with routinely available N95/P2 masks.
Footnotes
Author Contribution(s)
Acknowledgements
The author(s) thank the contributing staff members for sharing their fit-testing results and their enthusiastic participation.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Qualitative fit-testing kits were purchased by non-specific research funds of the investigators and the Department of Anaesthesia and Pain Medicine. Quantitative fit testing of anaesthetic staff (technicians and doctors) was partly sponsored by Avant Medical Indemnity Insurance (Sydney, New South Wales) and partly self-funded. Proshield N95 masks (TN01; BSN Medical, Mulgrave, Victoria) were made available by the institution. 3M N95 masks (9322A+P2; North Ryde, New South Wales) were donated by external organisations on compassionate grounds and by an independent occupational health and safety agency that conducted quantitative fit-testing (Occuhealth, Mandurah, Western Australia).
