Skip to Main Content
Contribute Try STAT+ Today

Editor’s note: The day after this First Opinion was published, STAT received by email a letter written by by Peter Marks, director of the FDA’s Center for Biologics Evaluation and Research, disputing the fact that the FDA used only one reviewer for each of three parts of its review of the Pfizer/BioNTech vaccine, as Peter Doshi and Matthew Herder suggest here. You can read the full FDA letter and the authors’ response to it here.

In what is arguably the most important decision the Food and Drug Administration has made this year — its emergency use authorization of the Pfizer/BioNTech Covid-19 vaccine — the agency apparently assigned only a single reviewer in each of two key scientific disciplines (clinical and statistics) to do the work in three weeks that usually takes months to do.

The FDA’s authorization last week followed similar authorizations in the United Kingdom and Canada. But the FDA’s decision is particularly important because of its reputation for being the international “gold standard” in regulatory rigor.

advertisement

Unlike its counterparts in other countries, the FDA is believed to be the only drug regulator in the world that consistently receives and reviews patient-level data from the clinical trials that underpin drug and vaccine approvals. To perform such rigorous analyses, the FDA typically spends around 10 months (a mere six months for applications given “priority review” designation) in an effort that involves reviews by experts representing various scientific disciplines: clinical medicine, statistics, pharmacology, chemistry, pharmacovigilance, and more. Together, these reviews form an “action package” which, by law, must be made publicly available 30 days after approval.

Given the urgency of the pandemic, the review of the Pfizer/BioNTech vaccine was conducted far faster than usual. The centerpiece of the analysis was data from the company’s 44,000-participant Phase 3 trial. FDA reviewers had just three weeks, from Nov. 20 to Dec. 11, to complete their analyses. It was a monumental task, which raises the question: Why didn’t the FDA devote additional reviewers to it? According to the FDA’s review memo, some scientific disciplines, such as pharmacovigilance, had multiple reviewers involved. But the two disciplines tasked with examining the clinical trial data and results, the clinical and statistical reviewers, were seemingly left to do their work solo. (Editor’s note: The FDA disputes this assertion in the letter below.)

advertisement

This seems wholly inadequate on at least two levels. First, without additional reviewers it is hard to comprehend how the work of several months could be squeezed into a matter of 22 days (including Saturdays and Sundays). In-depth review calls for examining patient-level data — a large feat that involves auditing and reviewing individual case records as well as independently rerunning analyses on the raw data.

Before the pandemic, it was typical to see just one clinical reviewer’s name for any given application. But given the stakes — and the time crunch — involved with reviewing Covid-19 vaccines, we would have thought the agency would do an even more thorough job than normal. But that does not appear to be the case.

One of us (P.D.) raised questions about potential unblinding in the trials through the vaccine’s side effects, as well as about the confounding effects of fever- and pain-reducing medications, which participants in the vaccine arm took three to four times more often than those in the placebo arm. Yet the FDA’s review shows no evidence that any of its scientists investigated either of these issues, and without more scientific staff devoted to the task it is hard to imagine how they could.

As one of us (M.H.) has investigated in the past with respect to FDA approvals not related to Covid-19, disagreements within the agency about whether to approve an intervention or set limitations upon its use are relatively common during the review process. Among 174 approvals examined between 2011 and 2015, 42 (24%) contained at least one disagreement among reviewers.

A key takeaway from this research is that a real strength of the FDA is its capacity to entertain dissent with a view to making better judgments about complex scientific evidence.

Such differing opinions were on display at last week’s FDA advisory committee meeting, where four advisers voted against the emergency use authorization that the FDA ultimately granted the next day. But that was the FDA’s external advisory committee.

Whether such disagreement also exists within the FDA is unclear. By seemingly assigning only one clinical, one statistical, and one toxicology scientist to review the Pfizer/BioNTech Covid-19 vaccine, it seems that discussion, let alone disagreement, was curtailed by design. The pressure on those lone FDA reviewers to do their work in record time, and do it without raising serious questions about the data, was likely immense.

Having an official second or third reviewer for all core scientific disciplines would have helped. Not making more reviewers available strikes us as an effective way to prevent in-depth assessment of the underlying data and limit the possibility of dissent. Shortcuts in the regulatory process undermine the very purpose of regulation to protect the public. If the goal was speed at all costs, we should just get rid of regulators. Otherwise, they need all the resources they can get to do their job, including the fast-approaching decision about whether to authorize Moderna’s Covid-19 vaccine.

Peter Doshi leads the Restoring Invisible and Abandoned Trials initiative and is an associate professor of pharmaceutical health services research at the University of Maryland School of Pharmacy. Matthew Herder is director of the Health Law Institute at the Schulich School of Law and an associate professor of pharmacology in the faculty of medicine, both at Dalhousie University in Canada.

Competing interests: Doshi has received grants from the FDA, and the Laura and John Arnold Foundation, and is an unpaid member of the Reagan-Udall Foundation for the FDA. Herder is a member of the Patented Medicine Prices Review Board, Canada’s national drug price regulator, and holds grants from the Canadian Institutes of Health Research.


To the Editor: The December 17 opinion from Peter Doshi and Matthew Herder is inaccurate and mischaracterizes the work of FDA career scientific staff involved in the review of the request for emergency use authorization (EUA) for the Pfizer-BioNTech Covid-19 vaccine. The authors’ opinion does not demonstrate an understanding of FDA’s team review processes that went into review of the EUA request.

Agency staff have been working around the clock for months, including nights, weekends, and holidays — long before the EUA request was submitted — providing feedback and advice to all sponsors developing Covid-19 vaccines. The authors cite FDA’s clinical review memo regarding the Pfizer-BioNTech Covid-19 EUA as evidence that more staff should have been assigned to conduct reviews. In fact, more than one hundred staff — including senior management from across FDA’s Center for Biologics Evaluation and Research, and FDA as a whole — contributed to this effort.

The authors clearly fail to understand that this is not the equivalent of an academic manuscript. The individuals listed are simply the leads for each review discipline, and do not represent the entire team of dedicated scientific review staff who worked tirelessly to provide thorough review of the EUA request.

The effort put forth by FDA staff to complete its review and authorize Pfizer-BioNTech’s Covid-19 vaccine for emergency use in 22 days following formal submission of the EUA request was heroic. This was not business as usual. FDA undertook an all-hands-on-deck approach to this work.

Because of the suffering caused by this pandemic, our career scientific review staff felt the responsibility to work through the review process with a tremendous sense of urgency while carefully doing their jobs to ensure that any authorized vaccine meets our rigorous scientific standards that Americans — and the world — have come to expect. To suggest otherwise is an affront to their incredible effort.

Peter Marks
Director, Center for Biologics Evaluation and Research
Food and Drug Administration

The authors respond: We appreciate Peter Marks’ response to our commentary raising questions about the rigor of the FDA’s review process for the Pfizer/BioNTech vaccine emergency use authorization. Marks faults us for taking at face value what is written in the agency’s review memo, which lists just one medical reviewer and one statistical reviewer as being involved in the review, which had a 22-day timeline from receipt of the application on Nov. 20 to the FDA’s decision on Dec. 11.

Marks says that the FDA staff has been working for months. We believe that. But it misses our concern that review of the results of the Phase 3 trial — which was completed just weeks ago and was submitted to the FDA on Nov. 20 — means the trial data were reviewed in three weeks, lightning speed compared to FDA’s normal months-long process. Marks provides no examples of how the underlying patient-level data were critically analyzed during that 22-day period. As our commentary suggests, we believe that a thorough examination of these data is not feasible within that time frame, at least not by a single clinical and statistical reviewer.

Regarding the depth of the FDA’s review, Marks says that more than 100 staffers “contributed to the effort” and that the individual names listed in the “Review Team” are “simply the leads for each review discipline.” We believe this, too, but wonder why two individuals are listed for pharmacovigilance and three for chemistry, manufacturing, and controls (CMC), but only one each is listed for clinical, biostatistics, and toxicology?

More importantly, as we explained, our concern is not simply about the quantity of reviewers — it is about quality of the process. We saw no sign that the agency assessed the possible impact of potential unblinding in the trial given the side-effects of the vaccine. And we wondered how many people were officially tasked with the responsibility of forming an independent view of the science. We remain concerned that there were no clear structural mechanisms by which reviewers who might have held dissenting views about the strengths and limitations of the application could freely document and discuss their concerns.

We are heartened to hear that the agency mustered “an all-hands-on-deck approach” to this review. It remains curious to us as to why the memo, combining all disciplines’ reviews, totals just 57 pages — much smaller than many solely clinical reviews. Ensuring that the memo accurately reflects all the work that went into the review, especially how and what was analyzed during the 22-day span, and lists all those involved in the effort is, in our estimation, the best course to building trust in the agency’s decision-making.

Peter Doshi and Matthew Herder

  • “The trial data on the vaccinations released so far have not addressed the issue of transmission of the virus. That is, the data are based only on symptoms and COVID-19 testing, not on transmission to others.”

    “Pfizer have no idea if the vaccine can prevent symptoms or transmission”

    • Polio vaccine doesn’t stop transmission either. I think there is clear evidence that both the Pfizer vaccine and the Moderna vaccine prevent symptoms based on the data. There is some evidence moderna decreases transmission but the researchers can figure that out while we get busy vaccinating

  • Here’s the problem with the vaccine at the moment

    “The trial data on the vaccinations released so far have not addressed the issue of transmission of the virus. That is, the data are based only on symptoms and COVID-19 testing, not on transmission to others.”

    https://www.americanthinker.com/articles/2020/12/covid_vaccination__warp_speed_ahead.html

    “Pfizer have no idea if the vaccine can prevent symptoms or transmission”

    https://www.conservativewoman.co.uk/more-serious-questions-about-the-pfizer-vaccine/

    • Again, the polio vaccine doesn’t decrease transmission. We get enough people vaccinated and protected, and it’s a moot point. Research on the vaccine is going to continue. In the mean time, I’m rolling up my sleeve for Wed 1 PM. I, and all of the docs I know, are eager to get the vaccine show on the road.

  • How did they get emergency use authorization ? There are viable alternatives.

    Hydroxychloroquine and Zinc, also Ivermectin

    • …perhaps because the items you mention are for treatment of illness whereas the vaccine is for prevention of illness… IJS…different category(ies)

    • “…perhaps because the items you mention are for treatment of illness whereas the vaccine is for prevention of illness… IJS…different category(ies)”

      Therein lies the problem , the vaccine data is only for symptoms not prevention or transmission

  • Thank you for this article. I have kept abreast of this Warp Speed agenda and have researched BioNTech and Moderna. Both companies have strong ties to the CDC and FDA’s research scientists and physicians. There is a lot of money to be made here by streamlining the roll out of the COVID-19 vaccine.

    • There are also a lot of lives to be saved by rolling out a vaccine. In addition, England has also approved the vaccine and the EU is about to in addition to many other countries. Every doc I know is rejoicing that we have a vaccine. Some are actually breaking down in to tears when they get it.

  • I understand the opinions of these authors but a few points: 1) Other international bodies similar to the FDA are independently coming to the same conclusions as the FDA. I’d be more concerned if there were disagreements 2) I’m not sure if the FDA has been given the resources they need by our current administration to do what these authors are suggesting. 3) Last week’s non-unanimous vote is being taken way out of context here. Having watching things live, there was debate about the 16-18 group and frankly the chair didn’t hand the process well. I highly suspect Pfizer at 18+ would have been unanimous as well 4) We had over 3000 people die in a single day this week. Deaths are rocketing past 300 K. Every day we wait for a vaccine with 95% efficacy is another potential death. Many of the physicians I know in the trenches would have loved to see this vaccine come out a month earlier.

    • Those 3000 deaths you mention died the same day? Or died in previous months, but lumped into the current reporting day? Please Give references to your statement.

    • Lots of sources for that but I’ve been looking at the Johns Hopkins data. If you go to 91 DIVOC and look at the 7 day average death curve for the US , our 7 day average is now above where we were in March/April. In my little corner of the medical world, we’ve had our highest hospital census and deaths since this all started.

  • Is the author sure that the FDA is the only agency that goes through patient level data when reviewing a drug/medicinal product application?

  • What is this? you asked for comments. I give you one and then you won’t post it censorship in America I thought we had a democracy.

    • Childish post. Do we really need posts like this right now? Why are you angry? You didn’t get your little question answered the way you wanted it to be? This is not your site it’s not mine . It belongs to the owners. if you don’t like that you are certainly welcome to go tweet that to the wind. Ungrateful people who stir up trouble are ugly no one needs that right now. Please leave your childish anger at the door, thank you.

  • Is the vaccine free to all that will get it. Could this be why their was not the staff that was needed to compleat the task at hand. Is there a mentality within the FDA at this time, that the company’s that are producing these vaccine can’t be sued,so just get it out the door. We did our part. It’s out of our hands now.

Comments are closed.