Quantifying Mask Leakage and Effectiveness for Public Health Messaging


 The purpose of this study is to compare masks (non-medical/fabric, surgical, and N95 respirators) on filtration efficiency, differential pressure, and leakage with the goal of providing evidence to improve public health messaging. Masks were tested on an anthropometric face filtration mount comparing both sealed and unsealed. Overall, surgical and N95 respirators provided significantly higher for filtration efficiency and differential pressure. Leakage comparisons are one of the most significant factors in mask efficiency. Higher weight and thicker fabric masks had significantly higher filtration efficiency. The findings of this study have important implications for communication and education regarding the use of masks to prevent the spread of COVID-19 and other respiratory illnesses specifically the differences between sealed and unsealed masks. One-Sentence Summary: The type and fabric of facial masks and whether a mask is sealed or unsealed has a significant impact on the effectiveness of a mask.


Introduction
Throughout the COVID-19 global pandemic, one of the most contested public health guidelines is the recommendation for wearing a non-medical fabric mask by the public. The Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO) have been responsible for managing shifting guidelines (1,2) as more knowledge from research regarding the pandemic is shared. The shift in messaging is understandably part of the reasoning behind why these guidelines have seen resistive adoption (3)(4)(5). Additionally, a lack of standardized test procedures for ltration e ciency of fabric masks or minimum performance requirements has severely impacted public messaging. In February 2021, in coordination with the CDC, ASTM International released F3502-21 Standard Speci cation for Barrier Face Coverings (6) as the rst standard to address non-medical fabric masks in response to the global pandemic. The intent of the standard is to provide testing standardization repeatability to improve communication and clarity for the public with respect to the ltration e ciency of barrier face coverings, more commonly known as non-medical fabric masks (6). In this study, the ASTM International testing procedure is applied using a face ltration mount to evaluate non-medical fabric masks, surgical masks, and N95 respirators in comparison with the newly established benchmarks. A unique assessment addition of critical importance for evaluating non-medical fabric mask e cacy is the evaluation of leakage impacts for all types of face coverings and the impacts on messaging to the public.
Barrier face coverings are de ned as a method of source control to reduce the number of aerosol droplets exhaled by an individual wearer and to provide some level of particulate ltration for aerosol droplets inhaled (1,6). Prior to the recent ltration e ciency testing standard (ASTM F3502), health organizations released guidance for health service professionals and the public on wearing face coverings. Generally, health organizations advised all persons who are not able to physically distance to wear a barrier face covering and those in medical care environments to wear certi ed N95 respirators or surgical masks (2,6,9). However, in many of these instances, proper tting and leakage minimization is not achieved leaving ltration e ciencies (FE) lower than the minimum requirement of 90% FE (7,8). While recent studies have examined the aerosol penetration of both at and structured materials (10)(11)(12)(13)(14)(15)(16)(17), variations across methods contributed to the need for standardized testing. A challenge for comprehensive and consistent testing is the inability to effectively measure a non-structured fabric mask, as opposed to the more rigid N95 respirator or at surgical mask. Therefore, prior studies' results are variable and may be misleading to the overall public understanding of mask ltration and effectiveness. A recent study following ASTM F3502 evaluated ltration of commercial fabric masks. Researchers developed a face mount apparatus based on the National Institute of Occupational Safety and Health (NIOSH) anthropometric head forms to test FE. When the masks were only sealed at the facial contact points, none of the masks in the study reported above the 20% FE minimum threshold (10). However, ASTM F3502 requires masks to be completely sealed around the edges to prevent leakage and more accurately measure the true FE for both in ow and out ow (6). In simplistic terms, the reality of mask wearers to properly seal all sides and edges of the mask completely to their face seems unlikely. Therefore, to use a standard that tests masks completely sealed to a xture is not in alignment with everyday application and minimizes the negative impact leakage can have on mask e cacy (10).
To date, the research remains limited on standard testing method outlined by ASTM International to evaluate (a) FE, (b) differential pressure (dP), and (c) leakage to compare non-medical masks with N95 respirators and surgical masks. In the aftermath of shifting public health recommendations on mask wearing, the lack of established research protocols for results dissemination signi cantly contributes to adoption variance (10,(17)(18). While many states and countries are lifting mask mandates, the CDC projects mask wearing through 2022, especially during u seasons. Without consistency in mask testing methods, the effectiveness of different mask coverings in mitigating the spread of COVID-19 cannot be effectively communicated to the public. Confusion about the effectiveness of masks, exacerbated by inconsistent recommendations early in the pandemic (19) may be associated with lack of adherence to masking recommendations, and inconsistencies in messaging about face masks foster mistrust of public health authorities (20). The public has expressed a desire for scienti c evidence on the effectiveness of masks (particularly cloth face coverings) to assist in decision making (20). Therefore, the purpose of this study is to evaluate mask coverings (non-medical/fabric, surgical, and N95 respirators) on FE, dP, and leakage with the goal of providing evidence to improve public health messaging. A major outcome of this study is to provide additional recommendations of communication in conjunction with the ASTM F3502 standards for proper quanti cation of leakage impacts when all masks are tested on an anthropometric head form in a Model 8118A Salt Aerosol Generator.

Face Barrier Selection
Per the recommendations provided by the CDC (1) and WHO (2) and prior face barrier ltration studies (7-16), researchers selected 13 face barriers, all containing multiple layers with the surgical and N95 respirators certi ed by a third-party and authorized for use in COVID-19 medical response environments.
All of the 11 nonmedical face barriers contained two or more layers, with F4 and F11 containing a polyester non-woven material for ltration. F1 and F6 are the only masks not commercially available and were constructed using a sewing pattern drafted from the headforms to maximize t. These two would represent the homemade face barrier options available. From the face barriers tested, seven, including the surgical and N95, contain 100% synthetic bers (polyester, polypropylene, nylon, and spandex); four contain a varying blend of natural and synthetic bers across layers (cotton, viscose, polyester, nylon, and spandex); and two contain 100% natural bers for all layers (cotton). Materials construction across all layers for nine of the face barriers tested are knit, knit/woven, or knit/non-woven combinations; two are woven, and two are non-woven (surgical and N95). See Table 1 (22). Researchers calculated material weight in grams per square meter (GM 2 ) using a 100 cm 2 sample cutter and followed procedures outlined in ASTM 3776 Standard Test Methods for Mass Per Unit Area (Weight) of Fabric (23). GM 2 was calculated for outer, inter, and lining layers (where applicable) and combined for a total GM 2 rating for each of the face barriers to be used for statistical analysis. Researchers followed ASTM D1777-96 Standard Test Method for Thickness of Textile Materials (24) to determine face barrier (all layers) thickness with a calibrated digital compression apparatus. See Table 1 for a complete breakdown of face barrier characteristics. procedural limitations for the current test method, researchers designed a face mount apparatus based on the NIOSH anthropometric data for standard head forms and sizes. The medium size head form was used and adapted to include a nasal passageway and open mouth with a hollowed interior to minimize the effect of results of face barrier evaluation. See Fig. 1. Preliminary results without a face barrier indicate the face mount apparatus effectively tests the ltration e ciency and differential pressure using the instrument and procedures outlined in 42 CFR Part 84 Standard Procedures (18) on ow rates and conditions for testing and certifying air-purifying and particulate respirators. Addition of the Face Mount Apparatus does not adversely affect the results for ltration or differential pressure. Results generated from aerosol production followed speci cations and standards outlined a Model 8118A Salt Aerosol Generator (NaCl). Figure 2 shows the face mount apparatus with a mesh addition and face barrier ttings which was used for the testing procedures.

Filtration Testing Methods
Specimens were pre-conditioned in accordance with ASTM F3502 Sect. 8.1.1.5 (3). Following preconditioning, specimen mounting, and setup followed two separate methods. Initially, specimens were mounted to the face mount apparatus and sealed only at the major points of facial contact: nose, chin and jawline (see Fig. 2). For the second round of testing all specimens were completely sealed and secured to the face mount apparatus. In both conditions, specimens were tested for in ow and out ow ltration e ciency and differential pressure. The face ltration mount was sealed to the ltration adapter plate using a hot melt glue. A cylindrical chamber was created around the device and sealed to the adapter plates. Face barrier testing was performed using the TSI 8130A with the chamber and face ltration mount. Testing procedures used ASTM F3502 (6) and 42 CFR Part 84 Standard Procedures (18) to set the ow rates and conditions for testing and certifying air-purifying and particulate respirators. Aerosol production followed speci cations and standards outlined with a Model 8118A Salt Aerosol Generator. Sodium aerosol (NaCl) speci cations included particles with a mass mean particle diameter of 0.26 µm and a count median particle diameter of 0.075 µm. Flow rates for both particle sizes followed standardized rates at ~ 85.0 L/min, much higher than the normal breathing rate of 40.0-60.0 L/min. Baseline readings were taken with the face ltration mount and no facemask installed. Results from baseline testing indicate minimal particle obstruction or distortion in the testing chamber for NaCl (0.08% FE, 0.0 dP). Means for FE and dP are used for statistical analysis as opposed to the lowest rates reported rounded to the nearest integer because the scope of this project is to not to certify any particular brand/type of face barrier.

Overall Mask Comparison -Sealed Mask Setting
Due to non-normality of the data, Kruskal-Wallis tests were conducted to determine if there were differences in outcomes across fabric types. For all tests, an alpha value of 0.05 was used. Post hoc analysis was conducted through a series of pairwise comparisons, with Bonferroni adjustment for multiple comparisons.
There was no signi cant difference found between surgical mask and N95 mask regarding ltration e ciency. With respect to the fabric masks, ltration e ciency was highest for fabrics 1 and 11, and lowest for fabrics 3, 4, 5, and 10. Other fabric types did not demonstrate a signi cant difference in post hoc analyses. Differential pressure was signi cantly different based on fabric type for both the out ow (χ 2 (11, N = 58) = 53.399, p < 0.001) and in ow (χ 2 (11, N = 58) = 52.559, p < 0.001) in experimental settings. Overall, differential pressure was highest for fabric masks 1, 6, and 9, and lowest for fabric masks 3 and 4. Again the surgical and N95 masks were higher than all of the fabric masks. Other fabric types did not demonstrate a signi cant difference in post hoc analyses.

Sealed versus Unsealed
Mask performance for the sealed condition was compared to the unsealed condition for the out ow condition using NaCl aerosol. Both ltration e ciency and differential pressure were signi cantly different based on whether the mask was sealed or unsealed on the experimental apparatus. Filtration e ciency (Table 1) was signi cantly higher in the sealed condition for fabric masks 1, 2, 3, 4, 6, 8, 9, and 11, as well as the surgical and N95 masks. Image 1a and 1b visualize the signi cant differences between sealed and unsealed for fabric masks and surgical and N95 masks.
Differential pressure ( Table 2) was signi cantly higher in the sealed condition for fabric masks 1, 2, 5, 6, 8, 9, 10, and 11, as well as the surgical and N95 masks, with the majority of differences realizing a large effect size. The differential pressure of fabric 3 was signi cantly higher in the unsealed experimental condition.  (Table 3): fabric mask 3, surgical mask, and N95 mask. All other fabric types showed no signi cant difference in ltration e ciency based on air ow direction. Differential pressure (Table 4) was signi cantly higher in the in ow condition for all fabric types, with the majority of differences realizing a large effect size.   Mask performance based on fabric mask type was compared for the sealed condition using NaCl aerosol. The masks were categorized based on weight (GSM -grams per square meter), thickness (mm), and ber (natural, synthetic, or blend). Results are shown in Table 5 (out ow condition) and Table 6 (in ow condition). Regarding fabric weight, fabrics with a higher weight has signi cantly higher ltration e ciency for both air ow conditions. Fabric thickness presented similar results, with signi cant higher thickness fabrics having signi cantly higher ltration e ciency measurements for both air ow conditions. There was no signi cant difference of fabric weight or thickness on differential pressure.
Regarding the type of ber used in the mask fabric, natural ber masks had signi cantly lower ltration e ciency than synthetic or blend masks. This was found for both air ow directions. When evaluating differential pressure, synthetic masks had signi cantly lower measurements than natural or blend masks. This was also found for both air ow directions.

Discussion
The ASTM F3502 standard test method to evaluate FE, dP, and leakage is outlined in this study and remains one of the earliest to follow the standard in the analysis of multiple types of masks as well as provide a comparison to non-standard mask ltration testing. Results from the testing for fabric masks, surgical masks, and N95 respirators are compared in terms of aerosol particle penetration (0.3 µm) with a NaCl solution at 85 L/min. For non-standardized testing, masks were tted to the face mount apparatus (see Image 1 in supplemental materials and methods) and adhered at contact points at the bridge of the nose, under the chin, and along the jawline for unsealed aerosol testing. In accordance with ASTM F3502, all masks were completely sealed to the face mount apparatus for standardized testing purposes around the edges using a double-sided adhesive. Masks were challenged with aerosol particles with the face mount facing up to evaluate the wearer ltration and exposure from external sources. The face ltration mount was inverted, and masks were challenged to mimic the wearer exhalation ltration. Both inhalation and exhalation testing were performed for both unsealed and sealed masks.  Table 4 indicates limited signi cant FE differences between in ow and out ow testing. Therefore, the FE data for masks is similar for both inhalation and exhalation, further supporting the compounding effect of both parties wearing a mask. However, there are signi cant differences in dP between in ow and out ow testing which is supported by the design of the face mount apparatus limiting exhalation out ow. Overall, this study con rms the validity of ASTM F3502 for evaluating sealed masks yet may not provide a real-world analysis of mask usage. This study identi es the crucial area of protection is in the reduction of leakage effects and supports proper secured tting to the wearer.
Although the rollout of certi cation of testing for masks is a step in the right direction for increasing public acceptance of usage, this study shows there are signi cant limitations for mask manufacturers and wearers to meet these new standards.
These results have important implications for communication and education efforts related to the use of masks to prevent the spread of COVID-19 and other respiratory illnesses. In particular, ndings related to differences between sealed and unsealed masks are of critical importance for health care workers. If a mask is not completely sealed around the edges of the wearer, FE for this personal protective equipment is misrepresented and may create a false sense of security. These results can inform efforts to educate health care workers and the public on the importance of proper mask t. Further, data on the comparative effectiveness of masks can be used to ful ll the previously established public need for evidence-based guidelines and inform future public health guidance and public communication efforts. Finally, to increase transparency and clarity, messaging on mask effectiveness for the general public needs to be crafted in line with the performance requirements outlined in ASTM F3502. Consistent use of these guidelines in communicating mask effectiveness may have implications for increased public trust of and support for public health guidelines.