Full vaccine results in, Pfizer/BioNTech stellar, but AstraZeneca/Oxford mixed & require more study

Today we received full data from both the Pfizer/BioNTech Phase 3 vaccine study and the Astrazeneca/Oxford vaccine trials that were reported only by press release several weeks ago.

Full Pfizer/BioNTech results:

First, the good news: The FDA transparently shared the agenda and full data report for the upcoming Thursday 12/10/20 meeting to consider emergency use approval (EUA) for the Pfizer/BioNTech SARS-CoV-2 vaccine.

This study was clean and straightforward: blinded randomized study of the vaccine given in two constant doses vs. a matched placebo, looking at symptomatic COVID-19 as the primary endpoint. Nearly all participants (>98%) received both doses, and on the scheduled timeline.

The demographics were not perfectly representative, but reasonably diverse (82% white, 10% black, 4% asian, and 26% listing Latino ethnicity), with 58% 18-55 years old, 20% between 55 and 65, 17% between 65 and 75, and 4%>75. There were a total of 46% who had comorbidities, with 35% reporting obesity.

In the primary efficacy analysis including 36,523 subjects, there were a total of 170 cases, with 162 in the placebo group and 8 in the vaccine group, for 95% efficacy (95% confidence interval 90.3-97.6). In the under 55 group, the efficacy was 95.6% and over 55 group, 93.7%, so it was efficacious for the older subjects not just the younger.

All sub groups defined by age, risk, sex, and race had similar efficacy, in the mid 90's%, so it worked equally well in all age, race, and risk groups.

Following is a plot of the cases as time since first vaccine dose -- note how both groups have equal case numbers in the first week, a time during which infection would have occurred just before study, but how the placebo group has cases increasing linearly, with the vaccine group almost completely flattening out once the second dose is given. These are remarkably strong results, indicating the vaccine prevented nearly all symptomatic cases.

This study doesn't have any information on asymptomatic cases since PCR tests were only given after reported symptoms, and there is little information on durability of immunity, although the horizon of the study going out 4+ months provides encouragement that immunity will be durable.

The study showed a low rate of severe adverse events, and most were comparable between the vaccine and placebo arms, except for higher proportions of fatigue, headache, muscle pain, or joint pain in the vaccine arms, amounting to discomfort in the day of the vaccination but nothing dangerous. Similar numbers of patients withdrew from the trial early because of experiencing adverse events, with 30 in the placebo arm and 37 in the vaccine arms. 6 of the 40,000+ subjects in the trial died during the trial, with 2 in the vaccine arm and 4 in the placebo arm, with no indication any were related to the vaccine in any way.

All together, these results show clear efficacy and safety, and warrant the FDA to approve and emergency use authorization (EUA) on Thursday. There is nothing controversial about these results -- they are clear and incontrovertible and scientifically sound.

Full Astrazeneca/Oxford results:

The full data corresponding to the press release of the Astrazeneca/Oxford vaccine are also now fully available today, with the peer reviewed paper published in the Lancet. The full protocols for the corresponding studies, with all amendments, is also available as Appendix 2 of the paper. The Moderna and Pfizer/BioNTech protocols were already publicly available, so now we have all details of the studies for all three vaccines.

Unfortunately, these results are not nearly as clean or studies as well designed as the Pfizer/BioNTech. These published results aggregate information across four different studies:

• COV001: phase 1-2 study in the UK (1077 healthy volunteers)

• COV002: phase 2-3 study in the UK (11,730 healthy volunteers)

• COV003: phase 3 trial in Brazil (10,002 healthy volunteers)

• COV005: phase 1-2 study in South Africa (2013 healthy volunteers)

This aggregated analysis was approved by regulatory agencies, but a meta-analysis across disparate trials is less informative than a single phase 3 study as performed by Pfizer/BioNTech and Moderna. Besides obvious geographic differences, there were also other key differences across studies including different placebos (meningitis vaccine for UK study, meningitis vaccine followed by saline placebo for Brazil, and saline placebo for South Africa), different age groups, different dosages, and different definitions of COVID-19 based symptoms.

In addition to these inconsistencies across studies, there are several other aspects of the studies that limit the usefulness of the information they provide:

• The trials were originally designed to just include a single shot, switched halfway through to include a second shot. Thus, there was great variability in when the second shot was given, from within 6 weeks for later participants up to more than 3 months for earlier participants.

• The studies were initially only including 18-55 year olds and health care workers, so initially included no older subjects and very few with co-morbidities. Later subjects included a small number of older subjects, but a vast majority of the data is only for young, relatively healthy subjects. Only 5 total COVID-19 cases were observed in participants >55yr across all 4 studies.

• A subset of early patients were accidentally given a 1/2 dose at their first shot, including 1367/11,730 of the UK study, and a handful of subjects in the Brazilian study. These were all between 18-55 and entered the study between May 31 and June 10, while the other full dose subjects were all after June 10 and included some older subjects.

• Given these irregularities, only 11,636/21,732 of the patients in the UK and Brazilian studies were available for the primary efficacy analysis.

The demographics of this trial are less diverse or representative than either the Moderna or Pfizer/BioNTech studies, with 82% between 18-55 years old, and 88% of the subset of subjects in the primary efficacy analysis. The UK study has 90-93% white, with the Brazilian study 65% white, 10% black, and 20% mixed race. The average BMI is 25.2, and there were few comorbidities, with 20-25% having diabetes, respiratory or cardiovascular conditions.

Getting to the results, for the primary efficacy analysis defined as symptomatic COVID-19 at least 14 days after second dose, which found a total of 30/5807 in the vaccine group and 101/5829 in the control group, for an efficacy of 70.4% (95% CI 54.8-80.6). The first plot below contains a plot of cases over time for this endpoint. The second plot includes the second efficacy analysis, which measures cases observed after 21 days past the first vaccine shot, which is important to look at given the inconsistency of timing from first to second shot. For this endpoint there were a total of 192 cases, with 51/6307 in the vaccine arm, 141/6297 in the control arm, for an efficacy of 64.1% (95% CI 50.5-73.9).

It is clear from these plots that the vaccine showed efficacy, reducing the number of symptomatic COVID-19 cases, but it is clear the efficacy is not nearly as impressive as seen in the same plot for the Pfizer/BioNTech study above, at least not for the combined analysis.

As mentioned in the press release, however, the small subset of the UK study accidentally only given 1/2 dose of the vaccine for their first shot did much better. In this group (labelled LD/SD in the plot below), there were 30/1374 cases in the control group but just 3/1367 in the vaccine group, for an efficacy of 90.0% (95% CI 67.4-97.0). This is on par with what was seen for the Pfizer/BioNTech vaccine above. Note that this group did not just get the lower dose at first shot, they also tended to have a longer time between first and second shots, which could partially explain the improved efficacy.

However, for the vast majority of the subjects in the study who received the full dose (SD/SD), which was the pre-planned protocol dose, 71/4455 control subjects were cases and 27/4440 vaccine subjects were cases, for an efficacy of just 62.1% (95% CI 41.0-75.7). If you look at these results split out by the UK and Brazil cohorts, here are the results:

Unlike the Pfizer/BioNTech and Moderna studies, the UK study also assessed asymptomatic disease. Subjects were given at-home test kits and asked to test themselves weekly throughout the study, with hopes of capturing asymptomatic cases missed by the primary endpoint. A subset of 6638 participants had valid data for this endpoint. Out of these results, 29/3288 in the vaccine group were cases, and 40/3350 in the control group were cases, for an efficacy of 27.3% (95% CI -17.2 to 54.9), meaning that apparently 27.3% of asymptomatic cases that would have happened sans vaccine were prevented by the vaccine. Split out based on dose, we see that for the subset mistakenly receiving the half dose, 17/1127 of control subjects were cases, and 7/1120 vaccinated subjects had asymptomatic disease, for an efficacy of 58.9% (95% CI 1.0-82.9), while for the full protocol dose subjects, 23/2223 of control subjects were asymptomatic cases while 22/2168 of vaccinated subjects had asymptomatic disease, for an efficacy of 3.8% (95% CI -72.4 to 46.3). These asymptomatic results may be affected by limited accuracy of the at-home tests, as well as the potential for these to capture residual mRNA from previous infections.

There were very severe cases in these studies, with just 18 hospitalized, 2 from the vaccine arm and 16 in the control arm, and only 2 severe cases, both in the placebo arm. Thus, these studies do not have much information on prevention of severe disease, which is not surprising given the focus on young, healthy subjects who have low risk of severe disease.

While the efficacy results were mixed, the safety analyses had stellar results. Serious adverse events only occurred in a total of 168 participants, 79 in the vaccine arms and 89 in the control arms. They did not summarize the proportions of non-serious adverse events across arms.

Conclusions

So, from these AstraZeneca studies, the positive takeaways are that the safety profile seems acceptable, with few serious adverse events, and fewer in the vaccine arm than control arm, and that the efficacy data shows clear reduction of symptomatic COVID-19 in the vaccine arm relative to the control arm, with especially strong results in the small subset that were accidentally given the half dose in the first shot. With this vaccine having roughly 10% of the cost of the Moderna and Pfizer/BioNTech mRNA vaccines and much easier to distribute, these results are encouraging.

However, there are numerous limitations in the AstraZeneca results, given the results combine information across disparate studies, involve mostly young healthy subjects, few minorities, and various protocol changes including addition of second shot after trial was half done, and a subset of subjects mistakenly receiving only a half-dose for their first shot. The relatively mediocre efficacy outside of this subset is disappointing, and the small sample size of the half-dose group means more data is required to validate those efficacy numbers.

The Pfizer/BioNTech study is much cleaner and covers a more diverse population, with exceptional efficacy results that are seen in all subgroups.

In my opinion, the data in the Pfizer/BioNTech study are sufficiently strong to justify EUA and immediate distribution, but I think that more data is needed from the AstraZeneca/Oxford before EUA is called for. The positive results are driven by the small subset mistakenly receiving the half-dose, but given this was not the protocol dose and only involves a few thousand subjects, it amounts to a single phase 2 study, which I don't find sufficient to warrant emergency use authorization. Hopefully more data will come in soon to provide support for the AstraZeneca/Oxford.