By Reuters Fact Check
Referring to a “peer reviewed study” published in medical journal The Lancet, users on social media have erroneously claimed that the reported efficacy rates for the available COVID-19 vaccines are “deceiving” and that the real rate of protection from immunization is much lower. This stems from a misinterpretation of two different measurements, the relative risk reduction (RRR) and the absolute risk reduction (ARR).
The posts feature a tweet that reads, “@TheLancet peer reviewed study confirms vaccine efficacy, not as 95% stated by the vaccine companies, but as: Astra Zeneca 1.3%, Moderna 1.2%, J&J 1.2% and Pfizer 0.84%. They deceived everyone by reporting Relative Risk Reduction (RRR) rather than Absolute Risk Reduction (ARR).”
Screengrabs of the tweet or posts with virtually the same wording have since spread to Instagram and Facebook.
A similar narrative is replicated in a TikTok video. “Receiving the COVID-19 vaccine reduces your risk at most 1.3%”, the women says to the camera around timestamp 00:33. “Why have we been hearing this vaccine has a 95% efficacy rate? Simple, they lied to you.”
The posts erroneously claim the article was a “peer reviewed study”, when it was actually a commentary by Piero Olliaro, Els Torreele and Michel Vaillant on April 20, featured in the Lancet Microbe.
When asked about the claim, Olliaro, professor of poverty related infectious diseases at the Centre for Tropical Medicine and Global Health of Oxford University told Reuters via email it was “extremely disappointing to see how information can be twisted.” He also said, “Bottom line: these vaccines are good public health interventions,” and added that in the commentary, “We do not say vaccines do not work.”
Statistical confusion
Posts refer to two statistical values in relation to how a vaccine impacts a population: Relative Risk Reduction (RRR) and Absolute Risk Reduction (ARR), which are calculated differently.
As explained by the National Collaborating Centre for Methods and Tools of McMaster University in Canada, these statistics “present the effects of an intervention in different ways and all provide useful information and together give a more complete understanding” of it.
According to medical experts at Meeden’s Health Desk, the RRR tells us how much the risk of infection is “reduced in the test vaccine group, compared to a control group who did not receive the test vaccine.” The RRR, or efficacy, tells us “how well the vaccine protects clinical trial participants from getting sick or getting very sick.”
This is what is usually presented as vaccine efficacy. For Pfizer BioNTech’s COVID-19 vaccine, this is 95%, for Moderna’s 94.1% and 66.3% for the J&J/Janssen vaccine.
The Absolute Risk Reduction (ARR) is the arithmetic difference between event rates (the percentage of people who, for example, got infected) within the two groups.
The Lancet commentary by Olliaro, Torreele and Vaillant shows the numbers for ARR, misleadingly referenced in social media posts were obtained.
Meedan Health Desk exemplifies how the ARR “will always appear low” as it depends on the event rate.
“Let’s say a study enrolled 20,000 patients into the control group and 20,000 in the vaccine group. In that study, 200 people in the control group got sick and 0 people in the vaccine group got sick. Even though the vaccine efficacy would be a whopping 100%, the ARR would show that vaccines reduce the absolute risk by just 1% (200/20,000= 1%). For the ARR to increase to 20% in our example study with a vaccine with 100% efficacy, 4,000 of the 20,000 people in the control group would have to get sick (4,000/20,000= 20%).”
Why RRR is used for vaccine efficacy
Natalie E. Dean, assistant professor of Biostatistics at the University of Florida, understood why the ARR numbers might have confused users on social media and explained why the RRR is the “usual scale” considered by the medical community when talking about vaccine efficacy.
“Because (the ARR) is a much lower number, it feels like it is saying that the other number (RRR) isn’t true,” but this is not accurate, “they are both capturing some aspect of reality, just measuring it in a different way,” she told Reuters via telephone.
Vaccine efficacy, expressed as the RRR means the vaccine will reduce the risk of infection by that reported percentage irrespective of the transmission setting. “It is more meaningful,” she said.
What studies say
Real world studies have already shown how the mRNA COVID-19 vaccines, for example, are highly effective.
Paul Offit, an infectious disease expert at the University of Pennsylvania who is also a member of the U.S. Food and Drug Administration’s vaccine advisory panel, pointed to two studies. A study by the U.S. Centers for Disease Control and Prevention (CDC) that found the risk of infection fell 90% two weeks after full vaccination with Pfizer or Moderna vaccines.
The other piece of real-world evidence highlighted by Offit was a Cleveland Clinic study, released in mid-May that showed 99.75% of patients hospitalized with COVID-19 between January 1 and April 13, 2021, were not fully vaccinated, reported by Axios here. “You significantly decrease the chances of hospitalization by being vaccinated. That’s a better way to look at it,” Offit noted.
Reuters has previously debunked posts on social media that push vaccine hesitancy.
Verdict
Misleading. The Relative Risk Reduction (RRR) and the Absolute Risk Reduction (ARR) are two measurements that are calculated differently. In terms of measuring how a vaccine impacts a population, they are complementary and not contradictory.
This article was produced by the Reuters Fact Check team. Read more about its fact-checking work at https://www.reuters.com/fact-check/about. For the original article with several links to the sources mentioned, including social media posts and statistics, visit Reuters.com.
