Masks: Paper shows 2020 mandates reduced spread, but that doesn't mean they are the right policy now

I have collaborated with a research team at PolicyLab at Children's Hospital of Philadelphia (CHOP) along with University of Pennsylvania that just published a paper in Health Affairs investigating whether the initial mask mandates in Summer 2020 made an impact local viral transmission and levels.

In this blog post, I summarize the results of this paper that mask mandates likely

conferred a benefit in reducing community transmission rates and case incidence during the

initial months of the COVID-19 pandemic, explain how its motivation was not driven by any political or policy agenda but a desire to learn what the data say, and also explain my personal viewpoint that these demonstrated effects do not imply that mask mandates are the right policy right now, especially for children in schools.

Motivation of paper:

This work was done by the PolicyLab COVID-19 modeling team.

Our PolicyLab COVID-19 modeling team has been active throughout the pandemic, first building models to track and project county-level cases counts starting in April 2020. This model, whose initial version was published in JAMA Network Open, modeled county level reproductive rate (i.e. increase of case counts) as a function of population density, seasonal weather factors, social distancing, recent case counts and testing positivity rate, as well as other factors. This model successfully predicted various local surges just before they happened, including the one in Houston, TX in early summer 2020.

The team coordinated weekly with White House Pandemic Response team leader Deborah Birx, who used these projections to help focus her efforts to identify hot spots and urge targeted mitigation in those areas to reduce the viral spread while minimizing the use of more drastic mitigation measures like lockdowns. Deborah Birx credited this work as mitigating COVID-19 spread, about which she said:

“I think that’s what every governor and mayor is looking at, but I really want to applaud that group (CHOP PolicyLab), because they really helped us think through this, and really take new ideas into the field,” she said. “Because you can’t keep locking down America, you have to have a way forward that brings common sense and practical measures together, that the American people can follow.”

Personally, this coincides with my own viewpoints expressed in this blog from the beginning of the pandemic that lockdowns were not feasible long-term, and that our primary focus should be sustainable targeted mitigation that reduced viral spread while minimizing the collateral damage caused by these mitigation strategies.

It was in the context of this work that our team set out in late 2020/early 2021 to empirically assess whether the mask mandates instituted in summer 2020, considered and promoted as an alternative less drastic mitigation strategy than lockdowns, had any effect on viral spread in counties after their initiation.

There was no political motivation in this work, and no desire to justify the mandates -- simply a desire to learn from the data through rigorous and complete modeling of the nation-wide county level data so we can objectively assess whether they had any impact.

Summary of Design and Analysis

The myriad factors affecting county-level viral spread make it difficult to assess the causal effect of a single factor such as implementation of mask mandates. Given that the locations implementing mask mandates have certain characteristics that potentially impact local viral transmission and community mitigation behavior, including high population density, recent viral surges, and specific political leanings, simply comparing masked and unmasked counties was not a legitimate modeling strategy. Such a comparison would be naive and rife with confounding that would bias the comparison.

Thus, our strategy was to used a county-level matched case-control design. Specifically, for each county enacting a mask mandate, we found a matched county without a mask mandate from the same region, but not the same metropolitan area, based on information from the Bureau of Economic Analysis. The matched county needed to have no equivalent masking mandate at least 3 weeks from the mandate county, and the match was defined based on population density, total population, election voting patterns in 2016, and case incidence and spread rate (Rt) in the two weeks before the initiation of the mandate, factors we found in our previous modeling to be largely influential on future case counts and thus potential confounders. The geographically closest county matching these criteria was chosen as a match. Because matches were defined based on timing of mandates, an "unmasked" county could be used as a control for multiple mandate counties, and could even appear later as a "mandate" county if a mandate was later instituted, in which case a suitable unmasked matching county would be selected. A list of all matched counties are provided in the supplementary materials of the paper.

This matched design adjusts for potential confounding based on these key factors. We added the election voting patterns not based on any political viewpoints, but as a surrogate for regional cultural factors affecting attitudes towards masks and level of voluntary masking.

As shown in the following table from the paper, this matching procedure effectively balanced these factors as well as other potential confounders, including socioeconomic status, diabetes %, social distancing measures, and seasonal factors. Diabetes % was a county-level measurement used as a surrogate for proportion of the population with co-morbidities, which can vary substantially across counties and impact COVID-19 incidence.

The effectiveness of this matching is what enables us to legitimately extract an estimate of the effect of the masking mandate on subsequent viral spread.

Our modeling used generalized linear mixed models with nonlinear smooth effects of disease transmission over time for each county, while adjusting for other confounding variables including social distancing at that time, population density, wet-bulb temperature at that time, and percent of county less than 2x federal poverty level.

Summary of Results

From this analysis, we estimated the ratio of COVID-19 case incidence per 100k people between counties with mandates and matched counties without. We can compute the more interpretable percent reduction of daily case counts between exposed and unexposed counties after intervention by taking:

% reduction = 100 x (1 - ratio)

We see that across all of the county matches, the COVID-19 cases were reduced by 8% by 2 weeks post mandate, 23% by 4 weeks, and 33% by 6 weeks, in counties implementing mandates relative to the matched counties without mandates. This is a substantial and statistically significant reduction suggesting that the mandates indeed had an effect on community transmission. After 6 weeks, the effect waned to 16%.

Based on this model, here is a plot of model-based estimates of the case counts under counterfactual scenarios assuming all counties had mandates or not, and demonstrates the substantial magnitude of the modeled effect.

Notably, when split by urban/rural settings, the benefit was strong in the more densely populated urban counties, with an 18% reduction at 2 weeks, 35% at 4 weeks, 48% at 6 weeks, and 27% after 6 weeks, while it was not evident in the rural/suburban counties at all.

Further, the effect was much stronger in counties voting Republican in the 2016 election, with a 17% reduction at 2 weeks, 31% at 4 weeks, 38% at 6 weeks, and 24% after 6 weeks, relative to Democrat-leaning counties with 3% reduction at 2 weeks, 16% reduction at 4 weeks, 30% at 6 weeks, and 11% after 6 weeks. This could reflect the prevailing attitudes towards masking -- that a higher proportion of the population was already masking pre-mandate in some counties than others, and it may be the Democrat-leaning counties with higher voluntary mask usage. This points out a key factor in this study -- that it is assessing the impact of mandates, but mandates only impact viral spread through behavior, and if the behavior (masking) is chosen voluntarily by many pre-mandate, the mandate may not have such a strong effect in that county.

In conclusion, this paper demonstrates through a matched analysis that on average, masking mandates conferred a benefit in reducing community transmission rates and case incidence in during the initial months of the COVID-19 pandemic. Although these benefits were not equally distributed, it appears that these mandates may offer broad value in reducing community risk during periods of elevated SARS-CoV-2 transmission in the USA.

Commentary: Does this mean that we should re-institute mask mandates?

The short answer is "no".

The timing of this paper's publication and the current political climate will likely obscure the main contribution of this paper, as many individuals tired of mask mandates, especially in schools, may be prone to dismiss it.

As mentioned above, the intent of the research was not to try to justify any particular policy or make any political point, but rather to objectively determine whether the mandates implemented in the early pandemic had any beneficial effect on community viral levels, using as rigorous and complete a design and modeling approach as our groups could devise to account for key potential confounding factors.

It is important to note what this paper did not do. It did not assess the effect of any specific type of mask on preventing individual infection for the wearer, or reduction of transmission to others from an infected wearer. It could not measure the effect of the level of actual mask wearing at the county level on viral transmission since we did not have access to any data on compliance. It simply assessed the effect of mandates, which could be attenuated by various factors, including non-compliance after mandates as well as level of mask-wearing pre-mandate.

What this paper demonstrated was that the mandates appeared to have a beneficial effect on reducing community spread in the early months of the pandemic, at least in the short term, with substantial reductions in the 6 weeks following the mandate, although waning a bit after that point.

This paper does not conclude that mask mandates should be continued indefinitely, should be implemented right now, and it certainly does not make any comments about whether they should be used in schools for children right now.

While our paper showed they made a practical difference early in the pandemic, this was during a time at which mask wearing was very rare in the community, so it was plausible that a county level mandate would increase the actual mask wearing behavior in that county. Further, it came at a time when the vast majority in the community were immune naive to SARS-CoV-2, with no previous exposure to the antigen to prime their immune system for rapid response if exposed. These factors likely contributed to the strength of the effects seen in this paper, and one might think the effects would be weaker if mandates were re-instituted now.

Mandates themselves can't directly impact viral spread, but only if they result in changed behavior, and that behavior reduces transmission. At this point of the pandemic, it is not clear whether mandates have any effect on behavior -- those who see value in masking will mask, and those who see no value will not.

Also, mandates have a sociological cost, exacerbating the divisiveness already present in society, tapping into deeply held political beliefs, making the issue a contentious emotional one. Indeed masking has become a controversial issue that has raised emotions on both sides, leading to distorted views at both extremes. The failure to wear masks is not paramount to assault as the emotional response by some against purported "anti-maskers" seems to imply, nor is it an existential assault on personal liberty that the outcry from some opposed to masking seems to imply.

As has often been the case in the pandemic, both extremes have a skewed perspective, and the pragmatic truth lies somewhere in between. This has been a major theme of this blog -- trying to find the middle ground between the extremes of "denial" and "alarmism" to identify the best course of action based on a careful assessment of available data, and this principle applies to masks as much as any other.

Masks should not be a big deal. Ample data and common sense suggest that masks (especially N95 or surgical masks) "help some" in reducing risk of exposure and transmission when properly worn, especially in crowded, enclosed indoor settings during times of high community infection levels. People should be encouraged to wear masks in such settings.

However, my opinion is that they should be voluntary, not mandatory, especially at this point of the pandemic in which nearly everyone's immune system has been exposed to the SARS-Cov-2 antigen in some form.

What about children? Should masks be mandated at schools?

Again, in my opinion, the short answer is "no."

Some feel strongly that masks should remain mandatory at schools. Part of this is driven by a natural and healthy instinct to protect our children from any and all harm, and part of it is driven by the school setting that is inherently different from the broader society. At school, children are together in enclosed and potentially poorly ventilated spaces all day long, and unlike in many general settings in the broader community, schools are able to readily enforce any mask mandate.

For this reason, many have continued to promote mask mandates in schools even after they have been eliminated in the broader local community.

In spite of the fact that mask wearing has reduced transmission at schools to some degree, is it beneficial or even reasonable to mandate it for children in school at this point of the pandemic?

Besides the fact that children have high previous infection levels, strong immune systems, and low risk of severe disease or death relative to adults, we also need to consider any collateral damage they might cause and consider long-term sustainability.

Masks have nowhere near the collateral damage of lockdowns, but in the long term many are concerned about the effect they might have on development of nonverbal communication in small children, and on social connectivity in general. We don't have data on the long-term effects of regular masking. They have now been worn daily in school going on two years, and for children in 2nd grade and earlier, they may have no recollection of mask-free schooling. How long should this continue?

Any child whose parents wanted them vaccinated could be vaccinated by now, and many others have been previously infected, so the level of immune protection among children is generally very high. Also, coming off the winter Omicron-driven SARS-CoV-2 surge, the community viral levels are declining in counties all around the USA, and in many have declined to low levels.

If we don't relax the school mask mandates now, when will we?

SARS-CoV-2 is not disappearing from the world any time soon, as we will likely see local surges for the next couple years at least, perhaps according to a somewhat predictable seasonal pattern. If not now, do we expect mask-wearing should become a permanent fixture of the school experience for children? Certainly not. Certainly I hope not.

I agree with the CDC’s recent recommendation of relaxing mask mandates and going mask optional at this time, especially in areas without high local transmission levels. To me, this decision was long overdue.

It may make sense to think about temporary measures to mitigate spread during massive surges of high local transmission, but I hope our divided society can reach a broad agreement that they should be relaxed and made voluntary when levels decline. A similar viewpoint was conveyed by my collaborator and senior author on this paper David Rubin, weeks ago in an interview on NBC.

In conclusion, this paper demonstrated that the mask mandates early in the pandemic appeared to have a beneficial effect in reducing community transmission and infection levels, especially in urban areas, and in counties in which masking was likely not broadly practiced pre-mandate.

However, this does not mean that mandates should be ongoing at this point of the pandemic, either in schools or more broadly in society. It is about time that we devise sustainable risk-based mitigation strategies, and hopefully we can eventually move past the contentious divisiveness that seems to characterize every decision made during this pandemic and devise a more united strategy for managing it.

Update 3/1/22: Revised analysis removing Yuma County

After the paper was published, another researcher pointed out a potential misclassification of one our unexposed matched counties (Yuma County). The process by which we classified all counties as having mask mandates or not was using publicly available records that we could source during a period that was highly dynamic. We have confirmed that indeed Yuma was misclassified as a county that did not have a mask mandate when in fact it did have a mask mandate since before the start of the study period. This county was used as a control for 26 masked counties, so affected 26 of the 328 mandate/non-mandate pairs.

We repeated the analysis with the 26 matched pairs involving Yuma county removed, and following is the updated table of results:

We see that the overall conclusions remain unchanged when removing the unclassified county, and in fact the magnitude of mask mandate effects get stronger.

The misclassification of mask mandate Yuma county to the non-masked mandate control served to bias our comparisons towards the null of no effect of mask mandate, and in fact the result is considerably stronger in the revised analysis, with the mask mandate leading to 14% at 2wk, 29% at 4wk, 41% at 6wk and 22% >6wk past initiating of the mandate, up from the previous analysis including the misclassified county with 8%, 23%, 33%, and 16% reduction at 2/4/6/>6wk, respectively.

Also, this correction substantially affected the suburban/rural results, that had little evidence of a mask mandate effect (-5%, 6%, -22%, and -5% change at 2/4/6/>6wk, respectively) in the original analysis, but -1%, 18%, 33%, and 9% change in the revised analysis with pairs involving Yuma taken out.

Here is the plot of the model-based estimates of mean infection rates per 100k under the counterfactual scenarios with all or none of the counties with mask mandates, respectively.

This demonstrates again that the magnitude of the mask mandate effect size in the revised analysis is greater than the original analysis that included the misclassified county.

Given the lack of central records of county-level mask mandates, we had to search through available public records and media reports to identify the places with mask mandate and initiation date, which raised the potential for us to misclassify a county as without a mask mandate if our search at the time missed reports of such a mandate. This is what appears to have happened with Yuma county. Fortunately, this type of misclassification if having any effect will tend to bias results towards the null of no mask mandate effect, as it did in this case, so did not induce spurious results. We are going back and reviewing all of our county classifications again with all of the information available now to ensure there are not any others that were similarly misclassified, after which we will re-do the matching based on the criteria specified in the paper and perform a fully revised analysis while re-matching (e.g. finding a non-masked match for Yuma and re-including these pairs in the analysis). I'll append the revised results here once we have them.