Skip to Main Content

In a U.S. pandemic response dominated by missteps, the effort to develop vaccines to prevent Covid-19 has so far been a triumph. Vaccines against the coronavirus that causes the disease are now racing through giant clinical trials as a result.

It’s a terrible irony, then, that the Trump administration’s statements have resulted in an erosion of public trust, with the percent of Americans who tell pollsters they would take a Covid-19 vaccine dropping and experts worrying the president could compel the Food and Drug Administration to approve a vaccine before one is ready. (Spoiler: No vaccine will likely be ready by Election Day.) 

“When the the president comes out and says, ‘by a very special day, we might have a vaccine,’ the whole thing blows up,” Ashish Jha, the dean of the Brown School of Public Health, said at a session focused on Covid-19 at the STAT Health Tech Summit. “In some ways, we’ve got to get the politicians to shut up and let the scientists talk about this and drive this process.”


The process of deciding when a vaccine appears to be safe and effective isn’t as straightforward as the general public might believe. But it’s important to understand it if we are to have confidence in these critical tools for helping to curb the pandemic.

Here, then, is a rundown of the science that goes into the decision-making process, what it tells us about when results could realistically be available, and when vaccines could start to be administered. This story is based on interviews as well as on documents the drug makers have released detailing their clinical trial plans. 


When will vaccine makers have enough data?

A clinical trial is typically sponsored by a company making a vaccine candidate or an academic institution, or a partnership of both. But it is actually monitored by what is known as a data and safety monitoring board, or DSMB, a group of independent experts hired to make sure volunteers in the study are safe. In many studies, the DSMB has the ability to recommend stopping a study not only if a treatment is unsafe, but also if it is so clearly effective that continuing just wouldn’t be ethical.

In the case of the vaccine trials, the studies being run by Moderna, AstraZeneca, and Johnson & Johnson with the National Institutes of Health share a common DSMB. The study being run by Pfizer and its partner BioNTech has its own.

The DSMBs will conduct what’s called an interim analysis after a certain number of people have been infected with Covid-19 and shown symptoms. Each of these cases is considered an “event,” and each vaccine maker has set a different number of events as a threshold to conduct an interim analysis as part of their trial protocols.

The study being run by Pfizer and its partner BioNTech, frontrunners in the race for a vaccine, is conducting its first interim analysis after 32 events, and would consider the vaccine effective if 26 people in the placebo group and six in its vaccine group had Covid. A study of Moderna’s vaccine, another frontrunner, is waiting until there are 53 cases of Covid.

In the case of Pfizer and BioNTech, an interim analysis could happen in October. 

Should a vaccine be approved, potentially for millions of people, after its efficacy has been shown based on 32 cases of Covid-19? 

Some experts say no. Eric Topol, the director of the Scripps Research Translational Institute, has been fervently saying that all the trials should continue beyond even their designed ends — when there are about 150 cases of Covid — saying that even the 150 number “may make statistical sense, but it defies common sense.” This could be particularly true if efficacy is limited, given that all the vaccines frequently cause side effects like fever.

Others say that while making a decision based on an interim analysis is fine, the first Pfizer analysis, in particular, seems to set a relatively low bar for efficacy given the small number of events.

The numbers are in line with past vaccine studies. Prevnar 13 was approved to prevent pneumonia in adults based on a study of 84,000 people that detected 139 cases of pneumonia, 90 of them in the placebo group. 

But there’s also a precedent for keeping such studies going to collect more safety data. Researchers studying RotaTeq, a vaccine to prevent a virus that causes childhood diarrhea, collected data from 70,000 patients to rule out a potential side effect which had been seen with a previous vaccine. 

The rules of approval

If and when a company believes its vaccine is safe and effective, it will then submit its data to the Food and Drug Administration.

No Covid-19 vaccine is likely to be fully approved by the FDA in the near term, because of requirements for manufacturing and follow-up that could take years. The FDA is expected instead to use a different authority by granting what is known as an emergency use authorization, or EUA.

The bar for an EUA is low, and past EUAs have seemed unwise in hindsight. A drug, peramivir, was authorized on an emergency basis to treat hospitalized patients during the H1N1 swine flu in 2009; the drug later failed to be shown effective in a clinical trial of such patients. Hydroxychloroquine was given an EUA to treat Covid-19 in hospitalized patients; that authorization was later rescinded once further study showed the drug did not benefit them. The EUA for convalescent plasma involved data that would never pass muster for an approval. These later examples are precisely why many experts are so worried that the FDA’s decision-making is politicized.

The challenge for the FDA will be to make sure that it brings its usual standards for a vaccine to the much more flexible emergency use authorization process.

Reviewing data on a drug candidate normally takes a year, six months if it is fast, and three months at the fastest. Even a truncated review should take weeks. So even if data on Pfizer’s vaccine are available in mid-October, an emergency authorization by Election Day is difficult to imagine. The same is true if data emerge from one of the studies of AstraZeneca’s vaccine being conducted outside the U.S.

That assumes, of course, that politics are not at play.

A Covid-19 vaccine is shown at a clinical trials site in Hollywood, Fla. CHANDAN KHANNA/AFP via Getty Images

Will the trials march on?

In the interim analyses that most people who follow medicine are used to, as soon as there is a clear result, the trial stops. But the plan for Covid-19 vaccines is different: Data from an interim analysis may be released if a vaccine is deemed inarguably effective — but volunteers may not be immediately told whether they are receiving vaccine or placebo. In other words, the study will remain “blinded.” Participants receiving a placebo will not be switched immediately to the vaccine. 

“The protocol is designed in a way that even if we would be able to file after an interim analysis, the protocol is designed to move on, at least for a certain amount of time,” said Kathrin Jansen, Pfizer’s head of vaccine research, on a recent call with reporters. The reason is that there is a need to assess efficacy in smaller groups, such as teenagers, the elderly, and people with HIV, she said. Jansen said that Pfizer and BioNTech hope to gather information about severe infection, too. 

Keeping a trial going after an interim analysis can be difficult, so much so that Thomas Fleming, one of the top minds in clinical trial statistics, co-authored a 2008 paper on why it should be avoided.

In a conference call with reporters to discuss the start of the study for Johnson & Johnson’s vaccine, Anthony Fauci, who heads research on infectious disease at the NIH, said that the other trials will continue in a blinded fashion until half the volunteers in the study have been followed for at least two months, in order to collect more data on efficacy and safety.

The J&J study requires only 20 cases of Covid-19 before an interim analysis is conducted. But Paul Stoffels, Johnson & Johnson’s chief scientific officer, told STAT that his company will wait to even conduct an interim analysis until after half the participants in its 60,000-volunteer study, started this month, have been followed for two months. Such an analysis will also require enough data in other populations, including the elderly. After the data are interpreted, the DSMB might recommend stopping the study.

“Generating enough data is also where we take our responsibility, because in the end we will be liable for the product getting into the market,” Stoffels said.

“You know, we go from a thousand people, to 60,000, to probably 100 million to 500 million people,” Stoffels said. “The information we generate has to be very solid.”

That’s the point that regulators and, especially, politicians need to remember no matter how eager they are to have a vaccine in hand.

  • You should clarify that the three studies require infection AND symptoms in the primary outcome. You could have the same number of infections in both arms but fewer symptoms in the vaccine arm and that situation would be called “effective”. Patients on the vaccine that become infected could then infect other people. We should look very carefully and the infectivity rate beyond symptomatology to determine the true value of the vaccines.

    • Hey Carlos,
      I agree with you however this is the test of immunity. Basically there should be mild to no symptoms in the vaccine group Vs normal distribution of covid in the other arm. However, 20-50 cases is very far from getting any statistical significance…

  • Unfortunately, not a great deal of insight or info is contained in this article. You did not spell out the parameters and meanings of Phase 1, Phase 2 and Phase 3 trials and what it takes to move a clinical trial through these phases. Laypeople have now heard these terms and what to know what they mean. Nor did you discuss what thresholds other drugs in the past have had to cross to make it to market (i.e. success rates). Poorly organized info. Scattered and inconcise. Disappointing.

  • Will we (the public) get to see the data? i.e. the numbers on infection/clinical illness events (as well as adverse events) in vaccine and placebo groups? Without that, widespread deployment of any vaccine should be resisted.

  • I am aged 73 and a MD. We should wait and listen to Science, not exactly to a group of scientists or a bunch of them, but the “whole” bunch of them. A vaccine does not have to be perfect. A person will not aprove any vaccine by himself, of course, but he keeps the right of deciding to take or not, one that’s available. If the cost/benefit analysis suggests a reasonable protection and safety of a vaccine, adjusting to my age group and health status’ needs, I would take it! I want to cut down my chances of having a severe disease, its possible permanent consequences or dying. 3rd of November to me is a mere coincidence. I am not American, I am not voting. An adequate vaccine in the middle of October, does not have to wait until middle of November to be given. What it’s gonna happen in a month’s time? We’ve seen it! We expect science to walk along common sense, otherwise dependability won’t be preserved. Do not politicize COVID-19 and vaccines! should be spelled from everyone to everybody else, just to be fair!

  • Why couldn’t the bar be set just a little bit higher with vaccines and EUAs.

    I think the politics needs to be kept out of it. I think all companies should wait till at least December if not, January to release the data. With no more elections in the way, it can be done objectively.

  • I wouldn’t buy any stock that Stat news is promoting. They are just paid propaganda. There record of being correct is less than a weatherman and there predictions. Hard to believe anyone would buy into anything they say.

  • I expected more insight on this matter from Statnews. Nothing insightful or informative here. I’ve read it all before. In fact, the article is less detailed with less information than I’ve read elsewhere.

Comments are closed.