Abstract
Background:
The SARS-CoV-2 pandemic put the entire healthcare sector under severe strain due to shortages of personal protection equipment. A large number of new filtering mask models were introduced on the market, claiming effectiveness that had undergone little or no objective and reliable verifications.
Methods and Materials:
Filter materials were tested against sodium chloride particles according to the EN149 §7.9.2 standard for particle penetration. Particle counters were used to measure the particle penetration of the filtering mask models, resolved over sizes in the range of 27–1000 nm.
Results:
We report on the results for 86 different filtering mask models. The majority of the tested models showed <3% penetration, whereas almost one third (i.e., 27 of 86) of the models performed poorly.
Discussion:
Interestingly, the poorest performing masks showed a tendency to have worse filtering effectiveness for larger particles than for smaller sized particles, following the opposite tendency of the best filtering masks.
Conclusion:
Almost one third of the filtering mask models tested failed the specified pass criteria as specified in the temporary EU COVID-19 standard. This fact, and the high health risks of COVID-19, highlights the need for independent testing.
Background
The response to the SARS-CoV-2 pandemic has put healthcare services in many countries under severe strain due to shortages of, for example, test kits, respirators, intensive care units, medical staff, and personal protection equipment.
The main infection routes for the virus are reportedly through droplet transmission (droplets in the size range 5000–10,000 nm) and fomite transmission, whereas aerosol transmission (particles in the size range <5000 nm) has been noted as a possible but less potent transmission route.1–5 As the counteractions have mainly focused on the risks of droplet transmission, serious concern that airborne transmission is underestimated has emerged. 6
The excretion of aerosols by human breathing, coughing, sneezing, or other activities is a complicated field.7,8 Furthermore, released aerosols are subject to many dynamic processes. For instance, water contained in aerosols evaporate quickly in air, which reduces the size of the residual material. 9 This process results in a transposition of the particle size distribution toward smaller aerosols, which, in turn, implies higher risk for airborne transmission. The SARS-CoV-2 virion is ∼100 nm in diameter,10,11 which acts as a lower limit for airborne transmission aerosols.
The dynamic processes have a profound impact on the transmission, which is manifested by the fact that the scientific discussion regarding the role of aerosols in disease transmission is dominated by the dynamics of the physical processes rather than the excretion process itself. A detailed review of the field is given by Jayaweera et al., 12 but is beyond the scope of this study.
It has been established that aerosol transmission is most prominent in case of enclosed spaces and prolonged exposures,3,4 for example, in hospital wards. 13 Preceding the COVID-19 pandemic, measurements that were conducted on the influenza virus indicated that humans may spread that virus through droplets with sizes down to 300 nm (possibly lower since 300 nm was the measurement limit in that study). 14 Further studies also suggest that the presence of aerosols well below 1000 nm can be expected.8,15 Indeed, it has been concluded that SARS-CoV-2 can be transmitted by airborne aerosols with sizes as small as the actual virion of ∼100 nm. 2
When looking at particle filtering materials, a number of filtration mechanisms are involved, such as impaction, diffusion, and electrostatic capture among others. The filtration mechanisms are not always overlapping in the size spectrum, causing filtering materials to have gaps between the dominating filtration mechanisms, which implies that particles of a certain size can penetrate to a higher extent compared with both larger and smaller particles. 16 Even so, it is commonly known that it is, in general, easier to filter out larger particles than smaller ones. For mechanical filtration, the most penetrating particle size is between 100 and 300 nm.
Where it has not been possible to maintain physical distance from infected individuals, in particular for healthcare workers, personal protection equipment is essential for providing a safe working environment. The sudden increase in demand for respiratory protection devices—on February 25 it was estimated that the USA alone would require 3.5bn N95 face masks during the pandemic for its healthcare workers, and only ∼1% of that amount being available at that point 17 —could not be met by the established suppliers' capacity. Hence, many new producers saw an opportunity to enter the market and boost the production of face masks. In the first half of 2020, 67,000 new companies registered in China that produce or trade in face masks. 18 Last year, China stood for half of the global production of 20bn masks, but this year China exported 70bn masks between March and May. 18
For a filtering mask to work properly the mask has to fit tightly to the face of the wearer and the material in the mask has to be porous enough to let air through, whereas filtering out as many particles as possible. To ensure that produced filtering masks meet the requirements, they undergo extensive tests according to national standardization procedures. The requirements include, for example, breathing resistance, total inward leakage, practical performance, and penetration of filter material. The valid standard for FFP2 (filtering facepiece) masks is EN149+A1:2009 in the European Union (EU), whereas in China and USA the valid standards are GB2626-2019 for KN95 masks and 42 Code of Federal Regulations (CFR) part 84 for N95 masks, respectively.
To address the acute shortage of filtering masks in the health system, EU introduced a temporary fast-track approval for SARS-CoV-2 use, by easing of the standardization process for filtering masks with penetration of filter material according to §7.9.2 in the EN149 standard as the only evaluation of filtering effectiveness where a maximum penetration of 6% for FFP2 masks are allowed. 19 This can be compared with §5.3 (GB2626) and §84.174 (42 CFR part 84) for KN95 and N95 masks allowing a maximum penetration of 5%. The question of facial fit was, however, left out to a qualitative assessment.
In the period April to June we, the Swedish Defence Research Agency, tested the filter material capacity of 86 different filtering masks (of EU and non-EU origin) for use in the health and care services in Sweden. The tests were mainly carried out as part of the temporary standardization process, but some were carried out as quality assurance at the direct request from healthcare providers in Sweden. Although we have probably not tested all filtering masks procured during the pandemic by the Swedish regions responsible for healthcare and municipalities responsible for care of elderly, we believe we have tested a large fraction of them and that our samples are representative of the filtering masks aimed for this sector in Sweden and EU.
In this article, we report on our test results showing that there is a large variation in filtration effectiveness between filtering masks—with the worst ones offering almost no protection.
Methods and Materials
All tested filtering masks consisted entirely of filter material, such as FFP2, KN95, or N95 masks. In this article, we present results exclusively from filter material penetration of sodium chloride aerosol according to the standard EN149+A1:2009 §7.9.2 with three samples as received for each filtering mask model supplied, meaning that all masks, also masks labeled KN95 or N95, were tested as FFP2 masks. However, we made a deviation from standard by using particle counters instead of the specified flame photometer, which allows for determination of particle size distribution.
The main components of the laboratory setup are schematically illustrated in Figure 1. The filter material penetration tests were carried out inside a test chamber, into which an aerosol containing a high concentration of sodium chloride particles was introduced. The resulting atmosphere is referred to as the challenge concentration, C1. After a stable challenge concentration was measured and monitored, the filtering mask under test was mounted in a sample holder and placed inside the test chamber. A flow through the filter material was initiated and the filter material was allowed to acclimatize for 3 min before the concentration of particle penetration through the filtering mask was measured in the filtered air flow, C2.
A schematic illustration of the main measurement components. Particles were generated and introduced into a test chamber obtaining a steady state challenge concentration, C1. This concentration was first diluted by a controlled factor and then measured and recorded by two particle counters. A filtering mask was mounted in a holder and placed inside the chamber where the particle concentration behind the filtering mask, C2, was led into particle counters and the concentration recorded.
The measurements were conducted at steady state conditions. The penetration was, with a few exceptions, measured consecutively with two different particle counters. The penetration percentage was calculated using Eq. (1) for each particle counter, either all particles or for a subset of particle sizes:
where Pp is the penetration percentage, C1 is the sodium chloride particle concentration in front of the filtering material, that is, the challenge concentration, and C2 is the sodium chloride particle concentration behind the filtering material.
A diluter was introduced into the measurement system between the test chamber and the particle counters. The high challenge concentration necessitated this complement to the measurement system to avoid saturation of the particle counters. The measured quantity for the challenge concentration, C1D, maps back to C1 by straightforward multiplication with the dilution factor DF.
The sodium chloride aerosol was created using a Collision Atomiser Type 4000 (SFP), filled with 2 wt% NaCl(aq) (CAS:7647-14-5; Merck) and operated at 2.1 bar with clean air, providing a polydispersed sodium chloride aerosol. Measurements showed that the particle sizes followed a lognormal distribution with a number median diameter of 200 nm and a geometric standard deviation (SD) of 1.8. This particle size distribution differs somewhat from specification according to standard EN149 (number median diameter of 60–100 nm and geometric SD of 2.0–3.0). However, this discrepancy has no influence on the results since this study measures the relative number of particles in front of the filter material and behind the filter material. That is, the absolute number of particles have no impact on the results assuming that there are sufficiently many particles to provide steady state conditions.
A total flow of 100 L/min containing the sodium chloride aerosol was led into the test chamber and thereby providing the challenge concentration. The challenge concentration was monitored and measured, after dilution, with two particle counters. The instruments used were LAS-X II (Laser Aerosol Spectrometer from Particle Measuring Systems) that is an optical particle counter and NanoScan (SMPS Nanoparticle Sizer 3910 from TSI) that is a condensation particle counter. The diluter was a 2-stage Dekati Diluter (DI-1000, operated at 2 bar).The concentration behind the filter was measured for 2–3 min on LAS-X II and thereafter 1–3 min on NanoScan, except when the penetration monitored during acclimation showed that the concentration exceeded the maximum limit for LAS-X II, then only NanoScan was used for measurements.
The dynamic range for LAS-X II is limited to 18,000 particles/mL at a sample flow of 10 mL/min, whereas NanoScan has a much larger dynamic range, up to 1,000,000 particles/mL. LAS-X II counts particles in the aerosol in the range 90–7500 nm, but we used 90–1000 nm arranged into 7 size channels to match the sizes of the generated particles.
The LAS-X II instrument was operated at a sample flow of 10 mL/min, averaging over time intervals of 30 s. NanoScan measures from 10 nm up to 420 nm, arranged in 13 different size channels (10 channels used in this article) with a sample flow of 750 mL/min, and a scan time of 45 s, which together with voltage ramp times give averaging time intervals of 60 s. See Table 1 for details regarding the particle size intervals captured by the different channels of the two instruments. The most penetrating particle sizes, that is, 100–300 nm for mechanical filtration, are included in this study that covers the particle size range of 27–1000 nm.
LAS, Laser Aerosol Spectrometer.
The presented work does not require ethical approval (no humans or animals participated as subjects in this study). No clinical trial has been conducted.
Results
Measurement Method Validation
The method of measurement described earlier was validated by a statistical analysis of the measured data set. The methodology used in this study assumes that steady state conditions apply, which means that measured values are ideally constant over the time of testing. However, some variation with time could be expected due to noise and imperfect experimental equipment. Furthermore, diversion from steady state may occur if dynamic changes of the filter materials during measurements are present. The latter phenomenon would constitute a physical property of the filtering masks, and thereby an interesting finding rather than a laboratory issue.
Measurements were conducted by collecting samples during time intervals of 30 or 60 s for LAS-X II and NanoScan, respectively. As already mentioned, the particles were separated into different channels depending on their sizes. The measured concentration in each such channel is constant over time in an ideally steady state system.
An important factor to consider is a possible systematic drift, that is, a consistent change in the measured mean values over time originating from faulty equipment or methods. To investigate this, we conducted the following analysis on LAS-X II data. For every filtering mask sample, the concentration for each time interval was divided with the time-averaged concentration to obtain a penetration fraction that may change over time. This was conducted for each channel individually. A consistent value of unity is optimal since it corresponds to a stable system. The mean of the penetration fractions for each channel was thereafter calculated for all filtering masks, which provides an insight into any systematic deviations.
The results are shown in the left panel of Figure 2. The penetration fractions were found to remain close to unity for the entire time period of 3 min. A moderate slope toward lower values of the penetration fraction with time can be detected. However, the changes are within a few percentages. Channel 7, which corresponds to 500–1000 nm particles, shows the greatest variations. The number of large particles in the measured interval is relatively small, due to the lognormal distribution of generated particles, resulting in increased uncertainty and less reliable statistics. The overall conclusion is that there is no significant systematic drift present.
The measured data were analyzed to detect any possible dependence on time during the 3 min that was typically used. For each filtering mask, the measured values for each channel over time were normalized by the corresponding time-averaged value. These relative values were collected for all filtering masks and the mean is displayed in the left panel. Any systematic drift in the measurement procedure would give rise to slopes in the curves. A similar analysis of the same data set is shown in the right panel. In this case, the absolute relative deviations from the mean values over time have been collected for each measurement. The means of these values for all filtering masks are shown with the purpose of exposing any strong fluctuating deviations that could be canceled out, or reduced, in the analysis displayed in the left panel.
Now, even in the absence of a systematic drift there might be considerable unbiased variances in the data. That is, the measured values may vary between time intervals, which indicate an unstable experimental setup. The right panel in Figure 2 shows a plot of the absolute relative penetration factor, that is, the deviation from the time average divided by the time average. This metric, ri,j, is calculated according to
where
The mean absolute relative penetration factors increase from ∼7% up to 23% depending on the channel number, that is, particle size. However, and more importantly, there is no evident change over time, implying that the steady state assumption applies to the filtering effectiveness during the measurement time period.
Another, but closely related, way of understanding these variations is by means of calculating the SDs over time. By dividing the SD for each measurement with the individual time mean values, for each channel, we obtained the relative variations of the measured values over time. This was conducted for all filtering mask samples and it was found that the SDs are within 10–18% of the time-averaged values for the lowest six channels, whereas the seventh channel had a value of 26%. It was found that the unbiased relative variations increased consistently with the particle sizes, which also can be formulated as a negative correlation with the number of particles present. This relation is attributed to the increased impact of noise at lower particle concentrations.
In conclusion, the system possessed low variations relative to the mean values, which indicate that the system was robust and that the measured penetration percentages constitute a reliable measure of the effectiveness of the filtering masks.
Particle filtering devices generally show better effectiveness over time, since particles clog the filter, and the test order could, therefore, influence the results. In this study, we measured the penetration using LAS-X II for 2–3 min first and conducted measurements on the NanoScan instrument for an additional 1–3 min. However, as concluded and indicated by Figure 2, no such tendencies could be seen during the measuring time used here. Only insignificant decreases in penetration percentages were detected, implying that the steady state assumption is valid under the present circumstances.
Model Series Variations
Comprehensive testing of a large number of samples from the same filtering mask model (and from the same producer) allowed for an intercomparison of filtering masks typically assumed to perform equally. One hundred thirty-eight filtering mask samples were used for this purpose, tested for quality assurance. For each sample, the time-averaged filtering effectiveness for each channel was calculated, that is, the penetration percentage. This means that we obtained 138 values for the filtering effectiveness for each channel. A box plot illustrating the distribution of these values is presented in the left panel in Figure 3.
Left, the variations within a series for one filtering mask model was investigated using LAS-X II on 138 samples. The boxes indicate the IQR, the whiskers relate to 1.5 IQR and the dots show outliers. Right, similar data set for another filtering mask model using 84 samples. IQR, interquartile range; LAS, Laser Aerosol Spectrometer.
For all channels it was found that the interquartile ranges (IQRs) are of the same magnitude as the mean values, indicating a sizeable variance between samples of the same filtering mask model. Furthermore, outliers are most frequent in the small particle size channels with considerable higher penetration percentage. For the smallest particles, some filtering masks had penetration factors 3–4 times higher than the median. As a complementary study, another filtering mask model was scrutinized with the same procedure. In this case, using 84 samples, the variation was somewhat smaller and there were fewer outliers. Both filtering mask models showed similar decline in penetration percentage with increasing particle sizes, that is, channels.
Performance Comparison of Different Filtering Mask Models
One metric of particular interest is the penetration percentage for all channels combined. To acquire this for a filtering mask model, the following procedure was conducted for every mask sample. The measured number of penetrating particles were summed over all channels and compared with the challenge concentration, yielding the penetration percentage. The penetration percentage lies in the interval 0–100, where 0 means that no particles penetrated the filter material, whereas a high value implies a poor filtering effectiveness. One value of the penetration percentage was obtained for each time interval. With a few exceptions, three samples were used for each filtering mask model. The same procedure was conducted for each sample, rendering a set of values of the penetration percentage. Finally, we calculated the mean of this set of penetration values, which serves as the final measure of the effectiveness for that particular filtering mask model.
The set of filtering masks used in this study contained 86 different filtering mask models. Poor effectiveness of 26 filtering mask models caused the LAS-X II instrument to saturate whereby no detailed quantitative data could be established in those cases. Saturation typically occurred when the penetration percentage exceeded ∼7% or 8%. The effectiveness was measured for each nonsaturating filtering mask model.
The distribution of filter effectiveness measure by LAS-X II is shown in the left panel of Figure 4. The majority of the masks have a penetration percentage <3%. Note that filtering masks that caused saturation are included with no particular effectiveness value (striped bar). The effectiveness of mask models that saturated LAS-X II was obtained by a similar procedure using NanoScan instead. The results for this subset of the filtering masks are shown in the right panel of Figure 4. As the striped bars indicate, there is a substantial number of filtering mask models with very poor effectiveness.
Distribution of total penetration percentage for the 86 filtering mask models tested. The striped bar in the left panel indicates the number of filtering mask models (26) that caused saturation of the LAS-X II instrument, implying poor filtering effectiveness. The right panel shows the penetration percentages on this subset of filtering mask models using the NanoScan instrument. Note that the two panels have different vertical scales.
The dependence of penetration percentage on particle size is an interesting property. To quantify this, the mean penetration percentage was calculated for each filtering mask sample divided into the channels. The mean over the mask samples was thereafter calculated for each channel. When this procedure was concluded for all 86 filtering mask models, we obtained a distribution of the filtering effectiveness over the channels. As mentioned earlier, 26 filtering mask models saturated the LAS-X II instrument. The same procedure was applied to this subset of filtering mask models using NanoScan instead. The resulting distributions are displayed in Figure 5 with LAS-X II and NanoScan results in the left and right panels, respectively. Note that the channels correspond to different particle size intervals for the two instruments (Table 1).
Left, the effectiveness of filtering masks resolved on the channels for LAS-X II. The filtering masks that caused the instrument to saturate (26 out of 86) are not included in this box plot. The corresponding data set for these filtering masks was instead acquired using the NanoScan instrument, and it is shown in the right panel. Note that there is a significant difference in the vertical scale between the two panels.
Discussion
Respiratory protection devices, such as filtering masks to protect against particles, consist of porous materials that are designed to let air through, whereas filtering out harmful particles, for example, dust, asbestos fibers, or, as in this case, the SARS-CoV-2 virus. The COVID-19 pandemic has exposed weaknesses with supply chains relying on just-in-time deliveries of personal protection equipment in the healthcare sector. As the crisis escalated, the supply side struggled to meet the overall demand, which leads to measures such as citizens' initiatives to produce homemade personal protection equipment, techniques to sterilize single-use filtering masks to open up for the possibility of reuse, 17 as well as new industrial producers entering a market of opportunities.
After the initial phase of acute shortage, the supply side managed to boost production to meet the demand. The capabilities of these masks were often undeclared or unreliable. The risk of exposing healthcare personnel to unintentional health risks called for comprehensive experimental testing of the equipment. This study has shown that even though most filtering mask models showed good results, the effectiveness of many filtering mask models were, indeed, found to be inadequate when subjected even to a fast-track standardized test protocol.
In EU, the European standard EN149 is valid, stating a maximum penetration of 6% for FFP2 filtering masks, which can be compared with maximum penetration of 5% for KN95 and N95 masks used in, for example, China (GB2626) and USA (42 CFR Part 84 §84.174). As we have shown in this study, many filtering mask models had a penetration percentage well above these maximum penetration limits, namely 27 out of 86 mask models tested in this study showed a particle penetration >6% and 29 mask models showed greater particle penetration than 5%.
For an individual wearing a mask, the protection offered is a combination of how well the mask fits to the face and the filtering properties of the filter material. Fit has been disregarded in this study since the work environment authorities in EU agreed to drop the quantitative test of total inward leakage (fit test) to reduce the time to market for filtering masks. The filtering effectiveness reported in this article is solely on the filtering material itself and corresponds, therefore, to a perfect fit, and constitutes an upper limit. Extra attention should always be paid to mask fit each time a mask is used. Major visual variations were noticed during testing, in particular the shape of the mask varied, which will affect the facial fit.
The experimental results presented in this study shows that filtering masks, as expected, have higher filtering effectiveness for larger particles and also that, although there is variance between individual samples, the variance decreases with increasing particle size (Figures 3 and 5). This is reassuring since it is likely that the virus will be contained in aerosols larger than the virion size of ∼100 nm.10,11 On a more surprising note, the subset of filtering masks that performed poorly and saturated the LAS-X II instrument showed the opposite by failing to filter out the largest particles during measurements with NanoScan. The detailed reason for this behavior remains unexplored.
The COVID-19 pandemic has revealed overlooked or neglected knowledge gaps regarding the physics of airborne transmission of diseases in general and the capabilities of personal protection equipment in particular. In this study, we have focused on the latter part and our intention is that the experimental findings provided may contribute to a better handling of the ongoing pandemic and improved preparedness for any possible future outbreak.
Conclusions
A large number of new filtering masks have appeared from a range of suppliers on the market during the COVID-19 pandemic. By extensive experimental testing, the effectiveness of 86 different filtering mask models was found to vary over the entire scale. A significant part of the models failed the governing international standards. In fact, 31% (i.e., 27 of 86) of the models failed the limit of maximum 6% penetration, whereas 34% (i.e., 29 of 86) failed the limit of maximum 5% penetration.
These results underline the importance of maintaining independent testing of personal protection equipment, despite acute shortages in supply during a pandemic, to avoid jeopardizing the health and safety of medical staff and, in turn, the population. Interestingly, a common property found among the filtering mask models that performed worst was that they particularly failed to filter out the largest particles, which was the opposite result compared with masks that passed the standard requirements.
Footnotes
Author Disclosure Statement
No competing financial interests exist. The data were collected from our tests of filtering masks. These tests were conducted on behalf of the national notified body (reporting to the Swedish work environment agency), or on behalf of the end users (healthcare providers)—and in both cases we act as an independent testing house.
Funding Information
No funding was received for this article.