Skip to Main Content

The drug maker Gilead Sciences released a bombshell two weeks ago: A study conducted by a U.S. government agency had found that the company’s experimental drug, remdesivir, was the first treatment shown to have even a small effect against Covid-19.

Behind that ray of hope, though, was one of the toughest quandaries in medicine: how to balance the need to rigorously test a new medicine for safety and effectiveness with the moral imperative to get patients a treatment that works as quickly as possible. At the heart of the decision about when to end the trial was a process that was — as is often in the case in clinical trials — by turns secretive and bureaucratic.

The National Institute of Allergy and Infectious Diseases has described to STAT in new detail how it made its fateful decision: to start giving remdesivir to patients who had been assigned to receive a placebo in the study, essentially limiting researchers’ ability to collect more data about whether the drug saves lives — something the study, called ACTT-1, suggests but does not prove. In the trial, 8% of the participants given remdesivir died, compared with 11.6% of the placebo group, a difference that was not statistically significant. 


A top NIAID official said he had no regrets about the decision. 

STAT Reports: STAT’s guide to interpreting clinical trial results

“There certainly was unanimity within the institute that this was the right thing to do,” said H. Clifford Lane, NIAID’s clinical director. “While I think there might’ve been some discussion, [because] everyone always tries to play devil’s advocate in these discussions, I think there was a pretty uniform opinion that this was what we should do.”


From the standpoint of the agency, he said, the study had answered the question it was designed to answer: The median time that hospitalized Covid-19 patients on remdesivir took to stop needing oxygen or exit the hospital was, at 11 days, four days shorter than those who were on placebo. “How many patients would we want to put at risk of dying,” he asked, for that last little bit of proof? Remdesivir, he noted, was not a home run, but is probably better than nothing.

Steven Nissen, a veteran trialist and cardiologist at the Cleveland Clinic, disagreed that giving placebo patients remdesivir was the right call. “I believe it is in society’s best interest to determine whether remdesivir can reduce mortality, and with the release of this information doing a placebo-controlled trial to determine if there is a mortality benefit will be very difficult,” he said. “The question is: Was there a route, or is there a route, to determine if the drug can prevent death?” The decision is “a lost opportunity,” he said.

Peter Bach, the director of the Center for Health Policy and Outcomes at Memorial Sloan Kettering Cancer Center, agreed with Nissen. “The core understanding of clinical research participation and clinical research conduct is we run the trial rigorously to provide the most accurate information about the right treatment,” he said. And that answer, he argued, should ideally have determined whether remdesivir saves lives.

The reason we have shut our whole society down, Bach said, is not to prevent Covid-19 patients from spending a few more days in the hospital. It is to prevent patients from dying. “Mortality is the right endpoint,” he said.

Most experts contacted by STAT expressed opinions that fell between Nissen and Lane, believing that the decision was a difficult case, with several defending the NIAID. 

“I think it was a really tough call,” said Janet Wittes, a prominent statistician and the president of Statistics Collaborative. 

When the remdesivir results were announced, the NIH said the data came from an “interim” analysis. This means that a study was stopped early because a drug’s benefit was so undeniable that it would be unethical to continue the study. But Lane said this was incorrect. The data come from a preliminary final analysis, a point at which the study would normally end.

The ACTT study (short for Adaptive Covid-19 Treatment Trial) began in late February. The first patient dosed in the study was an American repatriated from the Diamond Princess, a British cruise ship where there was an outbreak of more than 800 Covid-19 cases. By the terms of the study, hospitalized patients were randomly assigned to receive either intravenous remdesivir or a placebo. On day 15, the study would score patients on a scale from 1 (dead) to 8 (not hospitalized, with no restrictions on activities).

As results from other Covid-19 studies conducted in China started to trickle in, Lane and his team began to worry that looking at the outcome on only the 15th day could lead the study to fail even if the drug was effective. On March 22, with only 77 patients enrolled in the study, members of the NIAID team had a conference call on which they decided to change the measure that would be used. Instead of measuring patients on an eight-point scale on one day, the study would measure the time until the patients scored one of the best three outcomes on the scale. This decision was finalized on April 2; it was posted to, a government registry of clinical trials, on April 16.

Ironically, Lane said, the study would still have been positive if the change had not been made. But the change in the study’s main goal also changed the way the study would be analyzed. Now, the NIAID decided, the analysis would be calculated when 400 patients out of the 1,063 patients the study enrolled had recovered. If remdesivir turned out to be much more effective than expected, “interim” analyses would be conducted at a third and two-thirds that number.

The job of reviewing these analyses would fall to a committee of outside experts on what is known as an independent data and safety monitoring board, or DSMB. Though they generally go unseen, DSMBs are among the most important and powerful forces in medical research. They are allowed to analyze the data from a trial while it’s ongoing, even as drug companies, doctors, and patients are kept from knowing who is getting the medicine and who is getting placebo. These boards have two jobs: to make sure that patients aren’t being harmed by the experimental drug, and to ensure that it’s not already clear beyond a doubt that a medicine is effective.

Those decisions bring moments of triumph, despair, and, occasionally, confusion.

When Merck decided to withdraw the painkiller Vioxx in 2004, it was because a DSMB had recommended stopping a study of the drug when it became clear the medicine increased the risk of heart attacks and strokes. In 2014, when a study of the cancer immunotherapy Opdivo first proved that drug extended survival in melanoma, it was because a DSMB had found the result incontrovertible and recommended stopping the study.

But the DSMB for the remdesivir study did not ever meet for an interim efficacy analysis, Lane said. All patients had been enrolled by April 20. The data for a DSMB meeting was cut off on April 22. The DSMB met and, on April 27, it made a recommendation to the NIAID.

That recommendation was not about whether the patients on placebo should receive remdesivir. Instead, the DSMB recommended that in the next phase of the study, testing Eli Lilly’s arthritis drug Olumiant against remdesivir, there was no need for a placebo-only group. 

That decision, Lane said, led the NIAID to conclude that patients who had been given placebo should be offered remdesivir, something that started happening after April 28.

Learn how remdesivir works against the SARS-CoV-2 virus, which causes Covid-19. Alex Hogan / STAT

This is where Nissen and Bach disagree. There were 1,063 patients in the study, but only 480 had recovered at the time of the analysis. Researchers could have collected more data, they argue, and perhaps have learned if remdesivir saves lives. They were already close, both note. Results are considered “significant” if a measure called a p-value is less than 0.05; the value for mortality in the preliminary analysis was 0.059. “How many patients would we want to put at risk of dying to get that 0.01 on the p-value,” Lane retorted.

Marc Pfeffer, a cardiologist at the Brigham and Women’s Hospital in Boston, said he believes NIAID made the right call. He said that he was “very sympathetic” to the fact that researchers were getting this study done during a pandemic. “If you make the decision that remdesivir should be part of everybody’s therapy in the next phase, then those volunteers taking the risks in the current trial should be switched to the active therapy now considered effective,” he said.

Should this decision have been left to the DSMB, not the NIAID? DSMBs are technically only advisory panels, said Richard Chaisson, a professor at the Johns Hopkins Bloomberg School of Public Health. 

Chaisson remembers running an NIH-funded study of a preventative treatment for tuberculosis. The DSMB recommended continuing the trial, but he decided not to, because it was putting patients at too much risk. “The NIH had no problem with me not following the DSMB’s advice, and were even relieved I made the decision I did,” he said.

Wittes, of Statistics Collaborative, said she is glad she wasn’t on this DSMB, adding, “I don’t know where I would have come out.” And she said that when full results of the study are available, she would be “shocked” if the NIAID had not done things properly.

“I think there are groups of people who you’d really respect who would not have stopped a study like this without a mortality benefit,” Wittes said. “And I think you can argue that both ways.”

But she also worried that the evidence might not be strong enough to make the decision society is now making: that every new Covid-19 treatment must be given with or compared to remdesivir. 

“The danger is now it’s the treatment for everybody,” she said. “Now this is the base drug and everything is going to be that plus something or the control. I think we don’t know if it’s strong enough for it to be the standard of care. I don’t think we know who should be treated.”

Steven Joffe, an ethics expert at the University of Pennsylvania, said he believes the NIAID likely took the right steps in making its decision to give remdesivir to the placebo patients. But he worries about deciding to use time to improvement, not death, as the measure of success, in the first place.

“I don’t find this endpoint very compelling, and to me the real issue is the decision to design the trial around the endpoint of time to recovery defined in the way they defined recovery,” Joffe said. “To me, the decisions that are this weighty ought to be based on clinically important endpoints.”

All of this would normally wait until the full results were published, at which point the roster of the DSMB may be revealed. (Lane would not share their names.) But what is unusual in this case is that, before the data are even fully analyzed, the FDA has authorized remdesivir’s use. A Chinese study, meanwhile, failed to show remdesivir had a benefit. Several more studies of the drug are expected to read out soon.

Ethan Weiss, a cardiologist at the University of California, San Francisco, who traveled to New York two weeks ago to treat Covid-19 patients, said that he does worry that we have missed “a fleeting opportunity” to understand how well remdesivir works. “It is sad to me that we’re not going to get a complete answer about it.”  But he said he also thinks the issue is “inside baseball.” Remdesivir, as several experts have pointed out, is not a game changer.

The real problem, Weiss said, is not the handling of this particular study but that there aren’t more like it.  He said he wished the U.S. had built the infrastructure needed to do more studies like this when the pandemic in New York was at its height. He wished there were more studies, with more DSMBs.

“We’ve squandered an incredible opportunity to do good science,” Weiss said. “If we could ever go back and do something all over, it would be the infrastructure to actually learn something. Because we’re not learning enough.”

  • It looks like they turned a non-significant statistical result (the p value is insignificant) into a trivial triumph.
    What’s needed is avoidance of ARDS, even if a person has to spend a week or so extra in the hospital, it would certainly be worth it if they could avoid the development of ARDS, respirators, ECMO, permanent lung damage probably with oxygen requirements, ICU, renal failure-acute turning into chronic, dialysis, heart failure, cardiac arrhythmias. and cardiac death, along with some kind of undefined coagulation disorder which sounds ominously like DIC and the notorious cytokine storm.
    Have they avoided all of that?
    40 percent of fatalities in the US are people from nursing homes, who have failing overall health. It’s unrealistic to expect them to survive generalized organ failure.
    A medicine very early in the course, essentially prophylactic, to avoid all the complications of a fulminant viral infection is what’s needed.

  • If I were a medical student or resident or interested fellow or attending , particularly in NYC, I would do a chart review of a few thousand or all the Covid 19 patients and get the demographics and time lines of treatment and outcomes of the heterogeneous population of NYC, and you would have more data then anyone else. This would include the oral agents that were used as well as any compassionate use remdesivir. Let’s see what actually happened. Lots of people have been infected by the virus…..a study with 400 patients is not adequate .

    • ID Dr. studying patients outcomes in China did this and came up with the antacid famotidine. I’m all in favor of studying charts to look for clues but we need action now. Improvisation may not deliver a drug which can knockout the COVID virus in one go, but it may yield a pharmaceutical cocktail capable of safely mitigating symptoms and shortening recovery time.

  • When president Trump submitted that Hydroxychloroquine was a possible game changer in the war against COVID 19 , I applauded his aggressiveness and initiative in advancing something that might work . This was based on mostly anecdotal data, but it was worth a try for the most seriously afflicted patients. For all we knew, it could have been efficacious. The latest study released shows that it is not a game changer and in fact had fatally deleterious effects on some patients. I think Remdesivir is worth a try and I agree with a previous respondent that antiviral medications have been proven most effective when given early , not when waiting for a patient to need oxygen or worse, a ventilator. I think Remdesivir should be given to COVID patients early on to prevent exponential viral replication to the point of severe illness. No one can guarantee that it will work the way we are all hoping it will, but it is definitely worth a shot. Patients should be monitored closely for side effects . I am also in favor of combining another drug being studied now, Famitodine, for congruent therapy . Again, closely monitoring patients for adverse effects. If I , or someone in my family were gravely ill with COVID, this is the course of therapy I would select . What seems to have been forgotten in this crisis is that people need hope and positive initiative, not talk and debate. We can all debate the politics later, after we have defeated the enemy.
    God bless, and stay healthy

  • I am curious as to why the United States, United Kingdom, Canada, & Mexico have been left out of Gileads agreement to make Redemsivir available to those countries. Is one reason bc we have dropped our support of thr WHO?

  • Instead of paying $1,000.00 a dose why not take a pill that has proven same results or better for $.60 a pill? Not sure anyone’s mind thoughts or maybe blank thoughts!

  • Well, if Steven Nissen can honestly say he’d be ok with leaving one of his loved ones (wife, child, parent) on placebo “in the name of science” then his opinion carries weight. Something tells me he’d find a way to get HIS loved one on Remdesivir if they were on their death bed. Remdesivir is having a positive impact on those who are critically ill, meaning the virus has been in their system for a long time and has overwhelmed their immune system, I think it remarkable that an antiviral can have an impact at such a late stage. Let’s see how Remdesivir works if given much earlier. Also, can it be used prophylactically. If so, then it IS a game changer.

  • @Steve,

    Yes, of course trials are sometimes interrupted. They are interrupted frequently when harmful effects develop. And they are interrupted when there is clear benefit. But the operative word there is CLEAR. There needs to be overwhelming and obvious benefit.

    Neither improving from 15 to 11 days, which could very well be specious, and especially a p-value of 0.059 versus cutoff 0.05 are not CLEAR.

    I am particularly suspect of the former. Was the improvement in days one of the exit criteria determined ahead of the study? Or did they just happen to see that and jump on it? If the latter, they could, as well, be dredging. That is, if the same study was done, with a different population, this might not hold up.

    I think, as a physician recently noted in BMJ,

    “Expedited reviews or expedited publication are fine, but hinting that results are going to be positive only benefits the drug companies. It does not benefit the clinician. I, for example, would still not know if remdesivir was the right thing to give my patients.”

Comments are closed.