The question of whether and to what extent face masks work to prevent respiratory infections such as COVID and influenza has split the scientific community for decades.

Although there is strong evidence face masks significantly reduce transmission of such infections both in health-care settings and in the community, some experts do not agree.

An updated Cochrane Review published last week is the latest to suggest face masks don’t work in the community.

However there are problems with the review’s methodology and its underpinning assumptions about transmission.

The Cochrane Review combined randomised controlled trials (RCTs) using meta-analysis. RCTs test an intervention in one group and compare it with a “control” group that doesn’t receive the intervention or receives a different intervention. A meta-analysis pools the results of multiple studies.

This approach assumes (a) RCTs are the “best” evidence and (b) combining results from multiple RCTs will give you an average “effect size”.

But RCTs are only the undisputed gold standard for certain kinds of questions. For other questions, a mix of study designs is better. And RCTs should be combined in a meta-analysis only if they are all addressing the same research question in the same way.

Here are some reasons why the conclusions of this Cochrane Review are misleading.

It didn’t consider how COVID spreads and how masks work

COVID, along with influenza and many other respiratory diseases, is transmitted primarily through the air.

Respirators (such as N95s) are designed and regulated to prevent airborne infections by fitting closely to the face to prevent air leakage and by filtering out 95% or more of potential infectious particles.

In contrast, surgical masks are designed to prevent splatter of fluid on the face and are loose-fitting, causing unfiltered air to leak in through the gaps around the mask. The filtration of a surgical mask is not regulated.

In other words, respirators are designed for respiratory protection and cloth and surgical masks are not.

The review starts with an assumption that masks provide respiratory protection, which is flawed. An understanding of these differences should inform both studies and reviews of those studies.

The studies addressed quite different questions

A common mistake in meta-analysis is to combine apples and oranges. If apples work but oranges don’t, combining all studies in a single average figure may lead to the conclusion that apples do not work.

This Cochrane Review combined RCTs where face masks or respirators were worn part of the time (for example, when caring for patients with known COVID or influenza: “occasional” or “targeted” use) with RCTs where they were worn at all times (“continuous use”).

Because both SARS-CoV-2 and influenza viruses are airborne, an unmasked person could be infected anywhere in the building and even after an infectious patient has left the room, especially since some people have no symptoms while contagious.

Most RCTs of masks and N95s included in the review have not had a control arm – therefore finding no difference could indicate equal efficacy or equal inefficacy.

Studies examining wearing a surgical mask or respirator (such as an N95) only when in contact with sick people or when doing a high-risk procedure (occasional use) have generally shown that, when worn in this way, there is no difference.

An RCT comparing occasional versus continuous use of respirators in health care workers showed N95 respirators and surgical masks were equally ineffective when only worn occasionally by hospital workers. They had to wear them all the time at work to be protected.

We also combined only apples and apples in a meta-analysis of two RCTs conducted in exactly the same way and measuring the same interventions and outcomes. We found N95 respirators provide significant protection against respiratory infections when surgical masks did not, even against infections assumed to be “droplet spread”.

Most trials addressed only half the question

Face masks and respirators work in two ways: they protect the wearer from becoming infected and they prevent an infected wearer from spreading their germs to other people.

Most RCTs in this Cochrane Review looked only at the former scenario, not the latter. In other words, the researchers had asked people to wear masks and then tested to see if those people became infected.

A previous systematic review found face masks worn by sick people during an influenza epidemic reduced the risk of them transmitting the infection to family members or other carers. Preventing an infection in one person also prevents onward transmission to others within a closed setting, which means such RCTs should use a special method called “cluster randomisation” to account for this.

Data from a RCT of N95 respirator use by health workers showed even their unmasked colleagues were protected. Yet some of the trials included in the review did not use cluster randomisation.

The new paper combined health and community settings

This is another apples-plus-oranges issue. Different settings have widely differing risks of transmission, since airborne particles build up when sick patients are exhaling the virus in underventilated, crowded settings especially if many infected people are present (such as in a hospital).

A genuine protective effect of masks or respirators shown in a RCT in a high-risk setting will be obscured if that trial is combined in a meta-analysis with several other RCTs that were conducted in low-risk settings.

A large RCT in the community in Bangladesh found face masks reduced the risk of infection by 11% overall and 35% in people over 60 years. In contrast, in hospitals, N95 reduce risk by 67% against bacterial infections and 54% against viral infections.

Viruses like influenza also vary substantially from year to year – some years there is very little influenza, and if a RCT is conducted during such a year, it will not find enough infections to show a difference. The review failed to account for such seasonal effects.

But did they actually wear the mask?

The authors of the Cochrane Review acknowledged compliance with masking advice was poor in most studies. In the real world, we can’t force people to follow medical advice, so RCTs should be analysed on an “intention to treat” basis.

For example, people who are prescribed the active drug but who choose not to take it should not be shifted to the placebo group for the analysis. But if in a study of masking, most people don’t actually wear them, you can’t conclude that masks don’t work when the study shows no difference between the groups. You can only conclude that the mask advice didn’t work in this study.

There is a great deal of psychological evidence on why people do or don’t choose to comply with advice to mask and how to improve uptake. The science of masking needs to separate the impact of the mask itself from the impact of the advice to mask.

Mask-wearing goes up substantially to over 70% if there is an actual mandate in place.

It didn’t include other types of research

A comprehensive review of the evidence would also include other types of study besides RCTs. For example, a large systematic review of 172 various study designs, which included 25,697 patients with SARS-CoV-2, SARS, or MERS, concluded masks were effective in preventing transmission of respiratory viruses.

Well-designed real-world studies during the pandemic showed any mask reduces the risk of COVID transmission by 50–80%, with the highest protection offered by N95 respirators.

Many lab-based studies have shown respirators are superior to masks at preventing airborne respiratory infections and the incremental superiority from a single to two layered cloth mask to a three-layered surgical mask in blocking respiratory aerosols.

Yes, masks reduce the spread of COVID

There is strong and consistent evidence for the effectiveness of masks and (even more so) respirators in protecting against respiratory infections. Masks are an important protection against serious infections.

Current COVID vaccines protect against death and hospitalisation, but do not prevent infection well due to waning vaccine immunity and substantial immune escape from new variants.

A systematic review is only as good as the rigour it employs in combining similar studies of similar interventions, with similar measurement of outcomes. When very different studies of different interventions are combined, the results are not informative.

C Raina MacIntyre, Professor of Global Biosecurity, NHMRC Principal Research Fellow, Head, Biosecurity Program, Kirby Institute, UNSW Sydney; Abrar Ahmad Chughtai, Senior lecturer, UNSW Sydney; David Fisman, Professor in the Division of Epidemiology, University of Toronto, and Trish Greenhalgh, Professor of Primary Care Health Sciences, University of Oxford

This article is republished from The Conversation under a Creative Commons license. Read the original article.