Improved face mask testing system

Abstract

A new way to measure the penetration efficiency of a face mask is introduced. It has a number of advantages over the currently used measurement systems in which a stationary mask is impacted with a moving aerosol during which the upstream and downstream particle concentrations are compared. The equivalent system described here instead moves a mask through a stationary aerosol in a closed chamber and compares the particle concentrations before and after the movement of the mask. This system provides many advantages, such as greater simplicity, greater accuracy, an exact constant aerosol flow speed through a mask, more easily made concentration measurements, more realistic impacts, more controlled averaging, the absence of the need for preliminary impacts during flow stabilization, and a lower cost. A laboratory prototype of the system has been fabricated and seen to perform as expected. Data obtained from this prototype are displayed, analyzed, and used to demonstrate the consistency and repeatability of the system. A portable fully-automated compact prototype has also been designed and fabricated. This device can be easily operated, and the total time required to test a mask, from material insertion to result display, is only a few minutes.

Introduction

The importance of wearing face masks to reduce the transmission of airborne viruses such as COVID-19 is well-documented.^1–11 However, the CDC has reported that over 60% of the available face masks are poorly performing fraudulent products, often falsely labeled as being 95% efficient.^12–15 It is therefore very important to regularly test mask samples for compliance with efficiency standards such as the NIOSH standard 42 CFR 84, but currently this testing is expensive and prolonged and can only be performed by a limited number of laboratories using large and complex equipment operated by skilled personnel. ¹⁶ The utility of an accurate, easy to use, portable, and low-cost testing device can therefore not be overstated. This article will describe such a device and report on measurements made using it.

An effective testing device that measures the protective capability of face mask material must satisfy three conditions. (1) The test must be relevant. It must approximately replicate the way that the masks are used in practice. (2) The test must be accurate. The performance metrics must be measurable with a statistical error that is small enough to distinguish different objects, and must measure a mean value close to the true value. (3) The test must be repeatable. Measurements of identical objects, and re-measurements of the same object, must give the same metric value within the stated error range.

Considerable care is required to satisfy the above conditions while using the existing testing devices for measuring facemask filtration efficiency (ASTM standards F2299 and F3502, CDC-NIOSH standard 42 CFR 84, etc. The NIOSH standard stipulates the use of the TSI Model 8130 or 8130A Automated Filter Tester or an equivalent instrument.). In these protocols, efficiency is evaluated by measuring the percentage of encountered moving particles that are intercepted by the mask or mask material. This is done by impacting the the mask or material with an aerosol containing colloidal particles that are carried in air onto the mask or mask material at a specific airflow velocity, and measuring the particle concentration upstream from the mask or material (before) and also downstream (after). The penetration ratio R (the ratio of the downstream concentration to the upstream concentration) determines the filtration efficiency E = (1 – R)•100%. To effectively determine and compare efficiencies, the moving particle concentration and the upstream and downstream airflow velocities must be accurately measured and maintained, and the upstream and downstream particle counters must be accurately coordinated and be able to function in a moving air stream. This requires the incorporation of sophisticated and expensive laboratory equipment, including compressors, pumps, condensers, pressure reducers, flow valves, and flowmeters. A typical laboratory configuration is schematically illustrated in Figure 1, taken from ASTM Designation F2299.

Figure 1.

Diagram of conventional mask efficiency testing system, from ASTM F2299M.

Because of the complexity and cost of using the current mask efficiency measurement systems described above, it is extremely difficult for the user of a face mask to know if that mask is actually effective in providing protection from the inhalation of airborne pathogens present in his local environment. It has been widely reported that many commercially-available N95, KN95, and surgical masks are in fact almost-useless fraudulent products,^12–15 even if the mask carries a label from a reputable US manufacturer. Masks can currently be evaluated only by a limited number of laboratories using the current technology, and it can take weeks to receive test results, making the current testing logistics impractical for most people and organizations.

The alternative measurement system described here is based on the fact that it is much easier and more controllable to test a mask by moving it through a stationary aerosol. This greatly simplified testing technology will solve the current testing accessibility problem by enabling the placement of highly accurate, portable, inexpensive testing devices in any location with available electrical power. These devices will be simple to operate and can be used by almost anyone to determine if a given mask is sufficiently protective. The currently available testing devices are much too large, expensive, and difficult to operate and maintain for such widespread use (The operating manual for a conventional testing system runs over 245 pages.). In contrast, mask material can be easily tested by our device by placing it within a simple holder, screwing the holder together, inserting the holder unit into the device, and touching a start button on a touch screen. The penetration efficiency of the mask as a percentage will then be displayed on the screen within a few minutes.

Material and methods

New mask testing system

In this document the alternative, much simpler, and potentially more accurate, mask testing system will be introduced. It is argued that this (new) system is more relevant, consistent, effective, and repeatable than the current systems described above. The idea is to move a mask through a chamber containing a stationary particle-imbedded aerosol instead of moving the aerosol through a stationary mask. Other things (such as particle size and concentration) being equal, moving a mask through an aerosol chamber at a given speed is physically equivalent to moving the aerosol onto the mask at the same speed, and, assuming perfect accuracy, the mask efficiency measured either way will be exactly the same (Consider a stationary external subject observing a conventional mask test in which a stationary mask is impacted by an aerosol moving forward at speed v. If the subject is not stationary but instead moves forward at the same speed v, he will observe a stationary aerosol being impacted by a mask moving backward with speed v. Since the motion of the external observer cannot affect the efficiency of the mask being tested, the stationary mask test must record the same efficiency value as the stationary aerosol test. Likewise, if the observer moves forward at a slower speed u < v, he will observe an aerosol moving forward at speed v-u being impacted by a mask moving backward with speed u, and the measured efficiency will again be the same.).

Advantages

There are many advantages to using such a stationary aerosol system to test masks. These include:

1. The speed of a moving mask holder is easy to precisely set and maintain (e.g. by using an electric motor) and can be sustained with great accuracy. Measuring and maintaining an aerosol flow speed is, on the contrary, much more difficult, requiring pumps, control valves, flowmeters, etc. And even when using all this equipment a speed variation of ±2.5% is allowed.

2. In a moving aerosol, particle concentration and stability are difficult to maintain, requiring specialized valves and regulators. Constant concentration is automatic in a stationary aerosol.

3. Only one particle counter is required. The moving aerosol systems require two.

4. It is easier and more accurate to measure the particle concentration in a stationary aerosol than in a moving one.

5. The current systems simultaneously compare the impacting particle count (measured upstream of the mask) with the penetration particle count (measured downstream of the mask) even though the penetration particle volume came from a previously encountered impacting volume. In the new system the impacting particle count and the penetration particle count are made on the same chamber volume.

6. The current systems require that, before penetration data are taken, the mask material be impacted during a preparation time in order for the aerosol concentration to stabilize. This means that the testing does not measure the efficiency of a new mask but rather one that has been exposed for a period of time to a steady stream of particles. In the alternative system new mask material can be impacted immediately after the material is installed.

7. The current systems are expensive to purchase and maintain. The new system will be significantly less expensive and require only routine maintenance.

8. Current systems require the continuous introduction of salt particles during each test, whereas the new system requires only a one-time introduction of salt particles into the closed chamber. Only 0.01 g of NaCl in 100 ml of H₂O will be sufficient for about 500 tests.

9. Current systems provide outputs, typically stated as a three-digit percentage (e.g. 95.3%), obtained by averaging a sequence of measurements, with no indication of the spread of the individual measurements (standard deviation) or of the (systematic) accuracy errors (Random uncorrelated errors are reduced by averaging but systematic errors are not. The accuracy claimed by particle counters alone is ±5%.). Taking these spreads and inaccuracies into account, the scientifically appropriate way to present an output would include an error range (e.g. 95 ± 7%). The new system’s outputs will be in this scientifically correct form, if desired, but with smaller spreads because of increased accuracy.

10. With current systems, efficiency measurements are made on a mask exposed to a moving aerosol at a sequence of increasing times, and the average of these measurements is presented as the efficiency value. However, a measurement made at a later time will obviously take place after the mask has been exposed for a longer period of time, during which the mask will have absorbed additional particles, possibly changing (usually increasing) the efficiency value. This can introduce a systematic error into the measurements (made at different times) that cannot be reduced by averaging. This type of error is not present in the new system because the mask is impacted by the particles in the chamber only one time, and if averaging is used as an option to reduce random errors, the average is taken over measurements made on the same aerosol volume remaining after the movement of the mask.

A number of simplified mask testing systems have previously been proposed and operated,^12,17,18 but these all involve impacting a stationary mask with a moving aerosol, so they do not have the advantages listed above. They also lack the controls and instrumentation necessary for automated, accurate, repeatable, and statistically-significant mask efficiency measurements. Although possibly useful for detecting an ineffective mask, they cannot determine if a mask conforms to a published standard such as NIOSH 42 CFR 84. Such professional standards specify a precise aerosol flow speed (85 l/min), stability requirements, and calibration procedures that the referenced simplified systems do not provide. The new system described in this report has all of the listed advantages and can implement any published standard.

System operation

An example of the proposed system consists of a closed (say) horizontal cylindrical aerosol chamber within which is a concentric cylindrical shuttle that is effectively sealed but can slide back and forth. The shuttle has an open central cavity across which the mask material to be tested is attached, and has a low-friction coating, such as Teflon, as its outer surface. The shuttle is sufficiently long and close-fitting so that virtually none of encountered aerosol can escape around its outer surface as it slides through the chamber. The shuttle is propelled (say from the left end to right end of the chamber) by an electric motor that rotates a horizontal threaded rod that engages a matching threaded hole in the shuttle and terminates in a rotatable element within an end support attached to the left wall of the chamber. When the motor is engaged, the rotation of the attached rod causes the shuttle to move forward in the horizontal direction at a chosen speed. There are two external devices connected to the chamber, an aerosol generator and a particle counter. The concentration of salt in the aerosol generator is chosen such that the diameter of the generated salt particles has a chosen average value (The NIOSH 42 CFR 84 standard uses salt particles of median diameter 0.075 ± 0.020 µm.). The intake of the counter and the outlet of the generator are mounted flush with the chamber so that they do not interfere with the sliding motion of the shuttle. The system is illustrated schematically in Figure 2 and a prototype of the system, designed and fabricated by LexDesigns (Norwalk, CT 06855), is illustrated in Figures 3–5 (Details are given in the Prototype sections below.).

Figure 2.

Schematic drawing of new mask efficiency testing system.

Figure 3.

Illustration of prototype of new mask efficiency testing system. The shuttle moves from right to left.

Figure 4.

Illustration of prototype of new mask testing system with shuttle in motion, moving from right to left.

Figure 5.

Illustration of prototype of new mask efficiency testing system shuttle assembly.

The testing operation begins with the mask material clamped within the shuttle and with the shuttle at one end (say the left end) of the chamber. The aerosol, with embedded particles, is then released from the generator into the chamber until the particle counter indicates that the particle concentration is at the desired value N (particles per cc) (The aerosol drawn into the counter during the measurement should be returned to the chamber so that the concentration is not reduced. The ASTM standard F2299 requires concentrations N less than 102 particles/cc.). The motor is then engaged to move the shuttle forward so that the shuttle slides to the right at the desired speed until it is stopped at the right end of the chamber and the motor stops turning. The concentration N′ of the particles remaining to the left of the shuttle is then measured by the counter and used to evaluate the counting ratio C = N′/N. The fact that both the initial and final particle concentrations are made on a stationary aerosol is an important advantage of this system.

The most convenient design of the (say horizontal) shuttle cylinder places the mask material attached at the center of the left vertical circular face of the shuttle, with the shuttle otherwise empty, so that a cylindrical cavity exists between the clamped material at the left end and the open face at the right end (The prototype shuttle assembly is illustrated in Figure 5.). If the volume of this cavity is V0 and the total volume of the aerosol introduced into the chamber (including the open cavity) is V, then after the shuttle moves from the left end of the chamber and comes to rest at the right end, the volume of aerosol that impacted the material is V − V0 because the aerosol remaining in the cavity did not impact the mask. The number of particles that do impact the mask during the run is therefore n = N•(V − V0). The number of particles that penetrated the mask during the run is n′ = N′•(V − V0) because V − V0 is the total aerosol volume to the left of the shuttle corresponding to the concentration N′ measured at the end of the run. The penetration ratio is therefore R = n′/n = N′•(V − V0)/(N•(V − V0)) = N′/N = C, so that no correction has to be made for the volume of air remaining in the shuttle cavity after a run.

There is, however a different correction that may be required. It is possible that during a testing run a non-negligible number of the particles in the volume V − V0 do not impact the mask material or enter the shuttle cavity. If that is the case, the counting ratio C will not be equal to the penetration ratio R (the concentration of particles that penetrate the mask material divided by the concentration of particles that impact the mask material). The procedure for determining R from C follows.

Measurement corrections

There is a potential effect that could cause R to differ from C. As the shuttle moves (say from left to right) within the chamber, essentially all of the air in the chamber (initially to the right of the shuttle) moves into the cavity and, except for the volume V0, moves through the mask material and ends up at the left of the shuttle. However, it is possible that not all of the particles suspended in the aerosol volume V − V0 impact the mask as the shuttle moves forward. In general, the forward-facing area of the shuttle will include solid material (such as the area corresponding to the thickness of the shuttle wall and the surface of a clamp that holds a mask in place) in addition to mask material, and some of the aerosol particles may attach to this solid face material during a test. In addition, some particles may attach to the right-end wall or to other walls of the chamber. If these effects are not negligible, the penetration ratio R will not equal the (counting) ratio C of the measured particle concentrations in the chamber after and before the movement of the shuttle. The proposed correction procedure is described below.

In order to account for this possibility, the chamber is first filled with aerosol at a chosen concentration N (particles per cc) and the shuttle is then engaged to transverse the length of the chamber at a chosen speed v without a mask in place. The particle concentration N1 remaining in the chamber will then be measured and recorded (N1 < N is the concentration of particles that moved through the open mask area, and N − N1 is the concentration of particles that have not moved through the open mask area but have instead become attached to solid surfaces during the shuttle motion.). The shuttle will then be returned to the left-end starting position, the chamber will be cleaned, a mask to be tested will be attached to the shuttle, the particle concentration will again be increased to N, and the shuttle will again be engaged to transverse the length of the chamber at the speed v. The particle concentration N′ remaining in the chamber will then be measured (N′ < N is the concentration of particles that penetrated the mask.). Assuming that the same concentration N − N1 of particles will again attach to solid surfaces and not impact the mask as was the case when the shuttle moved without the mask in place, the penetration ratio (the number N′ of impacting particles that penetrate the mask divided by the total number N1 of impacting particles) can then be evaluated as R = N′/N1. Unless only a negligible number of the particles became attached to solid surfaces during the shuttle motion, this ratio will be greater than the counting ratio C = N′/N (R = N′/N1 = N′/N•N/N1 = C•N/N1.). In general, R will be greater than C, so that the filtration efficiency E = (1 − R)•100% will be less than the uncorrected value (1 − C)•100% (A similar correction should be considered in the conventional systems shown in Figure 1.).

The assumption that the amount of surface particle capture is the same when measured with and without a mask in place requires verification because of the possible existence of pressure differentials in front of and behind a mask in place (There is obviously no differential when a mask is not in place.). Because pressure measurement devices have not yet been incorporated into the new systems, this verification has not yet been made. However, in all of the completed testing, there was no measurable difference between the counting ratio C and the penetration ratio R, so that it was not necessary to make the corrections to C described above.

For chosen values of N and v, the correction factor R/C = N/N1 will be independent of the mask being tested and will therefore only have to be measured once. However, the factor may depend on N and/or v and, if so, it will have to be measured for each chosen value of these quantities. The penetration ratio R and the filtration efficiency E depend in general (through N1) on the geometry of the material holder and the chamber, as well as on measured particle concentrations. However, the setup shown in Figure 2 and the associated testing operation is obviously much simpler to assemble, operate, and regulate than the current systems shown in Figure 1, with potentially greater accuracy, and at a small fraction of the cost.

A second possible effect to be corrected for in the evaluation of penetration ratios from the counter outputs is the possible existence of a volume V1 of clean air behind a mask at the start of a test (If the material is attached at the left face of the shuttle, then V1 = 0 and no correction is necessary.). This can be accounted for as follows. Let V be the volume of air and let n be the number of particles to the right of the mask at the start of the test. The measured initial concentration is then N = n/V. Let n′ be the number of particles to the left of the mask at the end of the test. Then the measured final concentration is N’ = n′/(V + V1), the measured count ratio is C = N′/N = (n′/(V + V1))/(n/V), and the penetration ratio is

$$R=n\prime /n=N\prime •(V+V1)/(N•V)=C•(1+V1/V).$$

The penetration ratio is thus the measured count ratio multiplied by (1 + V1/V). Unless V1 = 0, R is therefore greater than C, making the penetration efficiency E = (1 − R)•100% less than the uncorrected value (1 − C)•100%.

If both of the above effects are relevant, the penetration ratio would be R = C•(N/N1)•(1 + V1/V). Depending on the location of the mask material within the shuttle, there could be a volume of aerosol remaining to the right of the material at the end of the test. It was shown previously that this does not affect the evaluation of R from C.

Additional details

The electric motor that propels the shuttle should incorporate a control mechanism that can be used to set the forward linear speed of the sliding ring to a desired value v. This value, and the value A of the cross-section of the chamber will determine the value of the aerosol flow rate f = v•A (f is specified as 85 l/min in the NIOSH standard, with an allowed variation of ±4 l/min (±4.7%). The variation in the new device is essentially 0% because, unlike the speed of a moving aerosol, the speed of an electric motor can be maintained with great accuracy.).

Even using the large array of required devices, the conventional testing systems involve a number of sources of measurement errors, but measurement inaccuracies associated with these errors are not disclosed in the provided literature that describes these systems. For example, no advertised particle counters state an accuracy less than ±5%, so the conventional use of two counters can lead to an error of ±10% (The new device described herein uses only one counter.). An additional potential error in the conventional systems is the uncertainty in the air flow rate (stated as ±2.5%). Increasing this rate decreases measured mask efficiency.^19–21 Also, errors can arise from the effects of particle accumulation before testing begins (As discussed in the NIOSH standard 42 CFR 84, this usually increases measured mask efficiency.). Other sources of potential errors have been listed above. None of these issues are present in the new system proposed here. In addition, conventional systems use undisclosed software to “reject bad data” and “implement accuracy requirements,” but the no quantitative measurement uncertainties are stated. Also, the stated efficiency value is the result of averaging over a number of measurements, but the spread in these measurements is not disclosed. The operation manuals state that increased sample times will improve accuracy and repeatability, but they do not state the degree of accuracy and repeatability that is actually provided.

The device shown in Figure 2 contains only the most basic components necessary for accurate and repeatable mask testing. Other components can however be added if desired or if required by a specified standard. For example, an aerosol dryer, a charge neutralizer, pressure transducers, in-line filters, and/or an external air compressor can be incorporated.

The size or composition of the particles embedded in the aerosol have not been specified here (The ASTM standard F2299 stipulates the use of latex spheres of diameter 0.1–5.0 µm.). The NIOSH 42 CFR 84 standard uses NaCl particles of median diameter 0.075 ± 0.020 µm. It has been argued that the most penetrating particle size is in the 0.2–0.3 µm range. ²¹ The new system described above can use particles of any reasonable size or composition. Specific particle counters or aerosol generators have also not been recommended here. Many accurate inexpensive models are commercially available. Mask standards (e.g. limits on filtration efficiency values such as 95%), or material properties (e.g. particle size, weight, composition, density, or speed), or controls (e.g. environmental or statistical) associated with the testing have also not been specified. These would be designated by test operators or appropriate government agencies (CDC, NIOSH, NIH, etc.). Note that in comparing the efficiencies of two different masks, it is important that all aspects of the testing equipment and conditions are identical.

Laboratory prototype

A prototype of the testing system described above has been designed and fabricated by LexDesigns (Norwalk, CT 06855). The device is illustrated in Figures 3–5 and a photograph is shown in Figure 6. The aerosol generator is the TSI model 3076 (TSI Inc. Shoreview, MN 55126) and the particle counter is the Met One model 804 (Met One Instruments, Inc., Grants Pass, OR 97526). In order to demonstrate the effectiveness of the concepts introduced here, this prototype has been used to measure the penetration efficiencies of various masks when impacted by salt particles of approximate diameter 0.3 µm (The most penetrating particle size is in the 0.2–0.3 µm range. ²¹ ). The measurements were made impacting a 3.25″ (8.255 cm) diameter circular area of mask material, with initial salt concentrations of approximately 15,000/l. The cylindrical chamber has a length of 36″ (91.44 cm) and an inner diameter of 7″ (17.78 cm), and the shuttle travels a distance of 30″ (76.2 cm) during a test. The salt aerosol flow rate is chosen to be 85 l/min, the value specified in the NIOSH standard (The NIOSH standard allows a flow rate variance of ±4 l/min (±4.7%), whereas the variance in this device is less than ±0.1%, the variance of the electric motor.). The corresponding shuttle speed v is 2.25 in/s (5.71 cm/s), which is the flow rate f divided by the area A of the chamber cross-section (f = A•v). The exposure time is therefore 13.35 s (The speed of the aerosol going through the smaller 3.25″ diameter mask material is increased to 10.43 in/s (26.49 cm/s).).

Figure 6.

Photograph of new mask testing system laboratory prototype.

Compact prototype

A portable fully automated compact prototype of the testing system has been designed and fabricated by LexDesigns (Norwalk, CT 06855). A circular section of mask material can be easily tested by this device by placing it within a simple circular holder, screwing the holder together, inserting the holder unit into the shuttle, and touching a start icon on an attached touch screen. The incorporated aerosol generator will then fill the chamber to a pre-determined salt particle concentration, and then an electric motor will propel the shuttle from the start position to the end position within the chamber. The concentration of the particles remaining in the chamber will then be automatically measured, and the penetration efficiency of the mask as a percentage will be displayed on the screen. The entire operation is completely automated, and the total testing time, from material insertion to result display, takes only a few minutes. In contrast, the operating procedure for the TSI Model 8130 requires 13 steps, with step 11 (setup) consisting of four sub-steps and step 12 (filter test) consisting of seven sub-steps.

Photographs of this prototype are shown in Figures 7 and 8. The device is fully self-contained with an incorporated air supply, emitter, sensor, and particle treatment components. The physical testing process is fully automated using an integrated touch screen that displays the sequence of testing operations and then displays the evaluated mask efficiency as a simple percentage value so that unskilled operators can easily run a test and understand the data output.

Figure 7.

Photograph of compact mask testing system prototype.

Figure 8.

Photograph of compact mask testing system prototype with covers removed.

Measurement results

Some of the measurement data obtained from the laboratory prototype are shown in Table 1. The tested material was from masks labeled as N95 and KN95, and from surgical (three ply) masks. The listed efficiencies are averages of six measurements and the evaluated, generally small, standard deviations are given in the adjacent column. The data displayed here are not meant to be a comprehensive investigation of the efficiencies of commercially available face masks. A relatively small number of mask materials were tested in order to provide a proof-of-concept and to demonstrate the viability of the testing system described herein (For a comprehensive review of mask efficiency measurements using conventional testing systems, see Howard et al. ¹¹ and references therein, and Joshi et al. ¹⁸ ). In the current tests, there was no measurable difference between the counting ratio C and the penetration ratio R so that it was not necessary to make the corrections to C described above. In other words, apart from the volume in the shuttle cavity, only a negligible number of aerosol particles failed to impact the mask as it traversed the chamber.

Table 1.

Test results for various face mask materials.

N95 mask	Average efficiency	Standard deviation	KN95 mask	Average efficiency	Standard deviation	Surgical mask	Average efficiency	Standard deviation
1	97.1	0.05	1	92.7	0.11	1	91.1	0.07
2	96.2	0.12	2	92.2	0.08	2	86.4	0.06
3	91.5	0.06	3	90.8	0.17	3	82.7	0.08
4	91.4	0.17	4	90.5	0.14	4	78.0	0.22
5	94.8	0.11	5	63.7	0.12	5	36.2	0.10

Discussion

The #1 N95 material had the highest measured efficiency rating of 97.1%. and the #2 at 96.2% was a close second. The #3 and #4 N95 materials were taken from the same mask in order to estimate of the variability of the testing results. The near equality of the results demonstrates the repeatability of the system. The #5 N95 material was tested using the ambient air in the laboratory instead of the salt aerosol in order to measure the efficiency of the mask in blocking 0.3 µm diameter particles in the local air. This measured 94.8% efficiency was relatively high. (A system that uses ambient air would be even simpler than the current one that uses salt aerosols, but the investigation of this possibility has not produced consistent measurements.).

The #1 KN95 material had the highest KN95 mask rating of 92.7% and the #2 at 92.2% was a close second. The #4 KN95 rating is from a repeat test of the #3 material and the results are again essentially identical, again demonstrating the repeatability of the system. The #5 KN95 material had the lowest KN95 mask rating of 63.7%. This low efficiency supports the by now well-known fact that the labeling of a commercially available mask as KN95 cannot always be trusted. The fact that none of the tested KN95 masks reached the claimed 95% efficiency level is surprising, but previous testes of these masks using conventional testing systems are expected to measure higher efficiencies because of the longer exposure times involved (See advantages 6 and 10 above.). Also, although the previous tests generally used the same 85 l/min salt aerosol flow rate used here, they often used different mask material sizes, aerosol particle sizes, and impact speeds.

The tabulated surgical masks had ratings in the 80%–90% range except for #5 which had an efficiency of only 36.2%, which effectively renders it almost useless. The #1 mask rating of 91.1% is higher than three of the KN95 mask ratings, and the #2–#4 mask ratings, although well below 95%, are large enough to provide important protection from Covid-19 pathogens.

The statistical significance of above data can be studied more rigorously and quantitively using an ANOVA program. The ANOVA results for mask comparisons K1-K2, K3- K4, and N3-N4 are displayed in Table 2. (1) Masks K1 and K2 were both labeled as KN95 but were made by different manufacturers and have slightly different measured average efficiencies (92.7% for K1 and 92.2% for K2). The ANOVA p-value (obtained from the Microsoft Excel program) for the combined data is extremely small (3.5•E(−6)), implying that the observed efficiency difference is statistically significant, with a confidence level near 100%. The conclusion is that identically labeled masks can have significantly different efficiencies. (2) Mask K4 is the same as mask K3 except that the mask material was new for the K3 test and had absorbed thousands of salt particles during the K3 test before the K4 test. The slightly different measured average efficiencies (90.8% for K3 and 90.6% for K4) suggest that the effect of the particle absorption was not significant, but the ANOVA p-value for the combined data is very small (0.007), implying that the observed efficiency difference is nevertheless statistically significant, with a confidence level over 99%. The conclusion is that particle absorption can significantly change mask properties. (3) Mask materials N3 and N4 are taken from different sections of the same mask, so it is expected that the small measured efficiency difference (91.5% for N3 and 91.4% for N4) is not statistically significant. This is confirmed by the large ANOVA p-value of 0.222. This is a strong confirmation of the repeatability of the new mask testing system: the measured efficiencies from different tests of the same mask material are statistically equivalent.

Table 2.

Penetration efficiency data anova single-factor study.

Efficiency data		Summary
Mask K1	Mask K2	Groups	Count	Sum	Average	Variance
KN95	KN95	Column 1	6	556.23	92.705	0.01375
92.61	92.03	Column 2	6	553	92.167	0.00691
92.56	92.14	ANOVA
92.66	92.15	Source of variation	SS	df	MS	F	p-Value	F crit
92.76	92.27	Between groups	0.86941	1	0.8694	84.177	3.5E-06	4.9646
92.76	92.23	Within groups	0.10328	10	0.0103
92.88	92.18	Total	0.97269	11
Efficiency data		Summary
Mask K3	Mask K4	Groups	Count	Sum	Average	Variance
KN95	rep of K3	Column 1	6	544.57	90.762	0.02854
90.66	90.47	Column 2	6	542.75	90.458	0.0193
90.69	90.61	ANOVA
90.69	90.61	Source of variation	SS	df	MS	F	p-Value	F crit
90.63	90.43	Between groups	0.27603	1	0.276	11.5415	0.0068	4.9646
90.82	90.38	Within groups	0.23917	10	0.0239
91.08	90.25	Total	0.5152	11
Efficiency data		Summary
Mask N3	Mask N4	Groups	Count	Sum	Average	Variance
N95	same as N3	Column 1	6	549.12	91.52	0.00436
91.58	91.17	Column 2	6	548.54	91.423	0.02859
91.55	91.29	ANOVA
91.58	91.47	Source of variation	SS	df	MS	F	p-Value	F crit
91.53	91.43	Between groups	0.02803	1	0.028	1.70174	0.22128	4.9646
91.46	91.55	Within groups	0.16473	10	0.0165
91.42	91.63	Total	0.19277	11

A comparison of efficiency measurements using the new verses the current testing methods is problematic because, as noted in advantages 6 and 8 above, the new method tests a new mask and the current methods test a mask that has been previously subjected to repeated impacts during “flow stabilization” followed by (averaging over) measurements made in a sequence of increasing times. To investigate this issue, the efficiency of a new KN95 mask was measured using the new system and the result was 90.3%. The mask efficiency was then measured a second time as 90.7%, a third time as 95.8%, and a fourth time as 97.1%. The increasing values measured in subsequent tests are expected because efficiency for this mask increases with increasing exposure time as more particles are absorbed (Mask Type 2 in the NIOSH standard.). A mask taken from the same package as this one was measured using a TSI model 8130 testing device by a government lab, and the reported result was 95.9%. This value is essentially the same as the 95.8% value obtained in the third test using the new system. Although this equality is gratifying, the fact is that the 95.8% efficiency is misleading because it is larger than the 90.3% value obtained when the mask was new. The value increases to 95.8% only after repeated exposures, so that a user of a mask from this package (labeled KN95) will assume that the mask is at least 95% efficient whereas it is likely to be only about 90% efficient when it is first used.

Although only 2 of the 15 efficiency measurements tabulated in Table 1 were significantly substandard, in fact over half of the masks tested in this study were found to be poorly performing. This alarming fact demonstrates the importance of inexpensive widespread testing of samples of all of the facemasks purchased by individuals or supplied to healthcare workers. The assumption that a mask labeled as being 95% effective is actually that effective can be a serious mistake.^12–15

Conclusions

Additional testing is required before the accuracy of the system described here can be definitively established, but the values of the mask efficiencies displayed in Table 1 demonstrate that the concepts introduced herein are effective in producing a mask testing system that is simpler to use and maintain then the systems currently in use. The acquired data suggest that the new testing system provides consistent and repeatable results, and has all of the advantages listed above. Because of the absence of a standard material with an efficiency value that is precisely known to within 1% or 2%, the accuracy of the system is not yet confirmed, but for the reasons stated above it is expected to exceed that of the systems currently in use. The small standard deviations associated with the measured efficiencies suggest that the results are realistic and consistent with a high accuracy. Furthermore, it is estimated that the cost of a commercial version of the system will be substantially less than that of currently available complete testing systems.

Because of financial constraints, the incorporated particle counter was limited to the detection of 0.3 µm or larger particles, but it is anticipated that more sophisticated dedicated particle counters and other components will be used in future testing. The system is currently capable of measuring mask material efficiencies only, but it would be easy to incorporate pressure sensors to determine the fit and breathability of the materials being tested. The prototype is currently capable of only measuring mask material that has been cut out of a mask, but a mask holder can be incorporated onto the forward face of the shuttle so that the resulting system can be used to test entire masks. A larger diameter chamber may be required for this purpose.

Recommendations

Given the well-documented widespread presence of poorly performing face masks on the market, often falsely labeled as being 95% efficient (see K95 mask # 5 and surgical mask # 5 in Table 1, and Bourouiba, ⁴ and references therein.), the utility of an accurate and low-cost testing device cannot be overstated. Although only 2 of the 15 efficiency measurements tabulated in Table 1 were significantly substandard, in fact over half of the masks tested in this study were found to be poorly performing to an alarming degree. The system described in this manuscript is sufficiently inexpensive and simple to use that the compact version of it would be suitable for on-premise testing. These devices could be widely deployed in hospitals, health clinics, doctor’s offices, drug stores, commercial and residential buildings, and even private residences, or anywhere there are people at risk.

The mask testing systems currently in use have been available for a long time, but they are very expensive and have not been subjected to comprehensive independent determinations of their accuracy. It is, in fact, currently impossible to determine the precise accuracy of any mask testing system because of the absence of a standard material with an efficiency value that is precisely known to within 1% or 2%. It is therefore highly recommended that the healthcare manufacturing community produce such standard material. In spite of this lack of quantitative accuracy confirmation, current systems provide outputs that are typically stated as a three-digit percentage (e.g. 95.3%), whereas a more scientifically honest way to present an output would include an error range (e.g. 95 ± 6%). Although the accuracy of neither the conventional systems nor the new system has been established, for the reasons given in the preceding sections the author expects that the new system will be the more accurate one.

In the absence of accuracy verification, it is important for the manufacturers of testing devices to at least confirm the repeatability and variance of measurements made using their systems. Repeatability can be verified, as was done here, by comparing the results of tests of material taken from the same mask, or at least from masks taken from the same container (Repeating a measurement on a given mask is not necessarily a reliable test for repeatability because the second test will have been made on a mask that differs from the original mask by having been impacted by a steady stream of particles.). Variance can be measured, as was done here, by evaluating the standard deviations (and/or other statistical metrics) of the data sets used in the averaging process. Independent verification of the repeatability, variance, and overall consistency of the penetration measurement systems used to test for compliance with official standards is essential to provide the public with efficient protection from airborne pathogens.

Research Data

sj-docx-1-mac-10.1177_00202940221137943 – Supplemental material for Improved face mask testing system

Supplemental material, sj-docx-1-mac-10.1177_00202940221137943 for Improved face mask testing system by Richard A Brandt in Measurement and Control

This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).