For weeks, the world has been eagerly awaiting clinical trial results for one experimental drug, remdesivir, to treat Covid-19. On some days, the entire stock market has moved up and down based on limited amounts of data about the therapy from Gilead Sciences.

The signals, so far, have been contradictory. An early peek at one study, based on data from patients treated at a Chicago hospital, suggested patients were doing better than expected on remdesivir. Days later, a summary of results from a study in China showed that patients on the drug did not improve more than those in a control group

A fuller picture on remdesivir is expected any day now. Gilead has said that it will release data from one of its U.S. trials — the one from which the Chicago results were disclosed — by the end of this week.  Even more data, from other trials including the China study, could follow shortly thereafter.

advertisement

But outside experts in clinical trial design worry that the results, instead of leading to a clear picture of whether the medicine is effective, will instead muddy the waters further.

The main concern, they say, stems from the fact that the Gilead trial expected to read out this week, which was conducted among patients with severe disease, lacks a control group — that is, patients who are randomly assigned to receive the best treatment available, but not remdesivir. As designed, the only randomization is the duration of treatment: either five days or 10 days of drug. Without a true control group of patients, many experts say, it will be difficult to determine whether remdesivir is effective.

advertisement

“The overall study itself has little or no scientific value since all patients are receiving the drug,” said Steven Nissen, the chief academic officer at the Cleveland Clinic and lead investigator of many trials for heart drugs that have been approved by the Food and Drug Administration.

“The study, as designed, is essentially useless and cannot be used by the FDA for consideration of remdesivir for approval to treat coronavirus,” Nissen said.

Peter Bach, director of the Center for Health Policy and Outcomes at Memorial Sloan Kettering Cancer Center, called the situation “frustrating.”

“For them to run the trial in severe but not include a control group, it’s just such a waste,” Bach said.

The predicament is symptomatic of one of the biggest problems in medicine: Collecting data quickly can actually slow things down if studies are not designed in a way that gives clear, definitive answers.

Not everyone is as grim about prospects for useful data. Scott Gottlieb, the former commissioner of the FDA and a fellow at the American Enterprise Institute, defended the Gilead study. He noted that while the trial expected to read out first is “open label”— meaning that both doctors and patients know who is getting the drug and who is not — more trials are still being conducted.

“There’s a rigorous, placebo-controlled trial that’s now fully enrolled and will read out soon, but there’s also a place for open label studies to broaden the safety database and answer discrete questions,” Gottlieb said, referring to a study sponsored by the National Institute for Allergy and Infectious Diseases.

Gilead says it had ‘only limited supply’

Gilead said that its study was never meant to be the first word on remdesivir’s efficacy. Instead, it was meant to follow two studies from China and the U.S. government study, run by NIAID.

“In the early stages of the pandemic, we not only had a limited supply of remdesivir but also a limited supply of the matched placebo required for placebo-controlled studies,” said Amy Flood, a Gilead spokesperson. “We chose to prioritize manufacturing active drug over placebo, and we provided our supply of placebo to China and NIAID for their studies of remdesivir.”

Given that three randomized, placebo-controlled trials were either underway or about to start, Flood said, Gilead designed its own studies to be open label. But this makes any conclusion from a study far weaker, because unconscious biases can affect the results.

Critics point to Gilead’s decision to compare two groups given remdesivir for either five days or 10 days. The problem with this strategy, they say, is that an ineffective drug that did nothing and a very effective drug that consistently helped patients overcome the virus would look the same in such a study. Only if the 10-day course were more effective, or if it was worse because of side effects, would the study have any clear result.

Gottlieb defended Gilead, arguing that increasing the size of the study from 400 patients to 6,000 expanded access to the drug at a time when it was needed to help with the public health emergency, but also allows the company to collect more rigorous data. “I think it’s far better to provide access in an open label setting than to merely give drug away in an expanded access protocol where you’re collecting minimal or no data,” he said.

Yet another trial in less sick patients, also run by Gilead, does have a control group and may give a clearer answer. Nissen sees “a reasonable study design.” But Bach was more critical, saying that even though that study has a control group, the lack of a placebo means the study might not be trustworthy. That’s because its main goal, time to improvement of symptoms, could be affected by the perceptions of clinicians and the patients themselves.

Bach said the hospitals conducting the study “are easily capable of wrapping syringes in brown paper and blinding the whole thing. I don’t understand why you would run a trial like this.”

The problem of placebo

When case numbers in the U.S. started to spike in mid-March, Northwell Health, New York’s largest health care provider, immediately staffed a 200-person unit to run Covid-19 clinical trials. Kevin Tracey, Northwell’s executive vice president of research, said that the group’s goal was to conduct rigorous clinical trials that included control groups. Why not have a placebo arm in the Gilead-run study of severe patients? 

“Obviously, that’s my first choice,” Tracey said. One reason it didn’t happen was that everyone needed to move very fast.

“In a quiet time when you have a few years to figure out the best way to lower cholesterol or the best way to treat arthritis, there’s a dialogue between the investigators and the company and the FDA and you negotiate the trial design,” Tracey said. “We didn’t do that with Gilead, in this case, because our hospitals were filling up with people who were dying.”

But Tracey points to another problem, too. He said that more severely affected patients and their families, having read optimistic news reports about a medicine, may not agree to be randomly assigned to placebo.

“That’s a reality,” he said. “You know that your mother, your wife, or your father, your brother is in the bed dying and you’re supposed to sign the papers that say you might get a placebo? No thank you, a lot of people say.”

At the same time, it can be hard to answer questions without randomized data, and even then results can be confusing.

What to expect from Gilead’s study

Gilead’s severe Covid-19 study is enrolling approximately 6,000 participants from 152 different clinical trial sites all over the world. The primary analysis to be disclosed this week is expected to encompass data from at least 400 patients, with a statistical comparison of patient improvement between the two remdesivir treatment arms — the five-day and 10-day treatment course. Improvement will be measured using a seven-point numerical scale that encompasses death (at worst) and discharge from hospital (best outcome), with various degrees of supplemental oxygen and intubation in between.

Video footage of a town hall at the University of Chicago, seen by STAT, provides some clues about what the results might bring. The researcher speaking in the video, Kathleen Mullane from the University of Chicago, said, “Most of our patients are severe and most of them are leaving at six days, so that tells us duration of therapy doesn’t have to be 10 days. We have very few that went out 10 days, maybe three.”

If Chicago’s experience holds true at the other hospitals participating in Gilead’s study, there will be little or no difference in patient improvement between the two remdesivir arms, making the comparison uninterpretable.

That, though, will not be the end of the conversation or the debate. 

Gilead could also provide additional analyses on the time to clinical improvement for different subsets of remdesivir patients, based on disease severity, onset of symptoms, or how quickly treatment was started. Without a control arm, these data will be hard to interpret on their own. 

One approach: compare to other trials, like the 199-patient study of the HIV drug lopinavir that was published in the New England Journal of Medicine in March. In the study conducted in China, lopinavir was 31% better than standard of care in time to clinical improvement, but the result was not statistically significant. Mortality was similar, with 19% of lopinavir-treated patients versus 25% of standard of care patients dead after one month.

Baird analyst Brian Skorney, for instance, points out that 79% of the lopinavir-treated patients were cured at one month, yet there was still no significant difference from the 70% cure rate in the control arm. This suggests that high rates of patient recovery should be expected, and that remdesivir — a modestly effective antiviral at best — will not make a big difference.

But RBC Capital biotech analyst Brian Abrahams suggests using the control arm from the lopinavir trials as an admittedly rough comparator when the Gilead data are released.

At seven days, 2% of the patients treated with standard of care showed a clinical improvement, increasing to 30% at day 14 and 70% at day 28. If remdesivir improves upon those results, perhaps doubling the time to clinical improvement at day 14, that might be seen as suggesting a benefit, Abrahams believes. 

Umer Raffat, an analyst at Evercore ISI, pushes back against the conclusion that lopinavir failed in the China study — which might bode well for remdesivir. Raffat points to an analysis in the NEJM study that showed in a small group of less sick patients treated before their symptoms worsened significantly, the lopinavir mortality rate was 45% lower.

Antiviral medicines should be more effective when given earlier. This is what Gilead is trying to do with remdesivir in its severe Covid-19 study. The company is expected to report mortality rates when it announces results later this week. If remdesivir lowers mortality to 10-15% from the mid-20% range reported in other studies, that might also suggest a benefit. 

Comparisons across different clinical trials may help paint a clearer picture of remdesivir’s efficacy, but they’re not likely to persuade FDA to approve the drug. That will only come from positive data in subsequent clinical trials. 

What to expect: the China-run severe study

The trial in severe patients conducted in China — the one accidentally posted on the WHO’s website last week — failed to show that remdesivir sped the time it took for patients to improve. After one month, it appeared 13.9% of the remdesivir patients had died compared to 12.8% of patients in the control arm. The difference was not statistically significant.

But the WHO did not post full data; rather, it was an abstract of the study and published without the full manuscript. It’s possible that the published version will be different.

It’s even possible this study will be supportive of some effect for remdesivir. The key is a statistic, known as a hazard ratio, showing that the time to clinical improvement for patients in one group was 23% more than the other. The key question is whether patients did better on remdesivir or placebo.

Raffat, the Evercore ISI analyst, obtained a copy of the study protocol, a plan for how it was going to be run, and wrote in a note to the bank’s clients that this figure was planned to be 1.4 if remdesivir were to have a statistically significant benefit. If that holds true, the 1.23 figure means that the study just missed showing the drug was effective, leaving hope that future studies could show the drug would be effective. 

“It’s not a silver bullet, but I do believe remdesivir is active,” Raffat said on a podcast for investors Friday.

The most clarity will come from the study from NIAID. It is controlled with a placebo, and patients and doctors are blinded to treatment. Full results are not expected until the end of May, but the study’s lead investigator, Andre Kalil, told Reuters on Friday that an interim look at the data may be available in a week or two.

  • Question for Steven Nissen and Peter Bach referenced above as critical of Gilead’s approach and the lack of a placedo control arm; if you were to get the Covid-19 virus today, what would be your plan of treatment? I’d love to know.

  • JC: we’re talking about patients in an ICU on a ventilator and the survival statistics for those patients are horrid. Yes, having a placebo controlled group is easier than rummaging through the million records we have of people who have contracted the virus and perhaps that control group doesn’t meet the definition of a perfect study. But the morality of the placebo approach is appalling. This is an example of a case where the perfect is the enemy of the good. Most people would not agree to participate in a study where the odds are 50/50 you will receive the medical equivalent of distilled water and if you do, the odds are 80% you’ll die. Wouldn’t it be more moral to search the records and find very good matches who were not given the drug to be tested in the study group? We have the data, why not use it?

    • I could not agree more. There may be some efficacy to drugs like remdesirvir or lopanivir or others either singly or in combination. We need tests of different combinations of drugs and different sequences of drugs at different stages.
      I agree that the ethics of condemning people to 80% mortality with a placebo in order to have better statistical data are appalling. Why to throw data away or to disregard it when peoples’ lives are at stake?

  • all very much like skating on thin ice. How dare anyone publish this unproven content!

  • Dr. Nissen is a little overly pessimistic. There are ethical issues in using placebo groups in emergency conditions to start with, which is why most end-stage cancer treatment trials are performed in salvage conditions. Further, it is entirely possible to match the Chicago patients post hoc to controls. Not perfect, no, not a controlled clinical trial, no, but far from “useless”. Rigor is ultimately essential but lets not throw the baby out with the clinical trial bath water. Right now we know squat about how to proceed. Any data should be squeezed from any trial to learn whatever the heck we can.

  • We know doing nothing doesn’t work and that’s what a placebo is — doing nothing. How can we morally initiate a study where we deliberately do not treat people in an ICU on a ventilator when the likely result is that 80% will die? Even if searching the records for a control group doesn’t meet the standards of perfection an “ideal study” calls for, isn’t that good enough?

    • That is Dr. Paul Marik’s argument when treating septic shock patients using historic controls when defending his IV Vitamin C, hydrocortisone, thiamine recipe. It reduced mortality from > 40% to 9%. How can he then go back and do a double blind study?

    • Well said, Lauren. Why do these people want to see Gilead fail? The Chicago trial is getting great results.

    • Actually most patients who go into the hospital leave the hospital alive. It’s not AIDS or pancreatic cancer. So saying “Nothing works” is wrong.

      Or more technically it’s correct in that doing nothing works very often.
      The question is does doing something else work even more often.

      We can’t know that without concurrent placebos controls.

  • In the early days of the onset of the virus in the U.S., the hospitals had nothing to use to treat the disease. Several hundred people died. That group could form the basis for the placebo control arm. There is no need for a placebo control arm.

    • Exactly my point but statistically we need to take account of differences in regions related either to differences in the capacity of health care systems or the virulence of the different strains of virus. If the death rate in one region is lower than another (neither of which used a therapy) that needs to be taken into account.

  • If they had limited supply isn’t that MORE reason to do a placebo-controlled trial? They’d need less of their product if 1/2 or 1/3 got a placebo.

  • Everything will be inconclusive until the results are conclusive. A test amongst 5,000 patients is a good start, but not conclusive.

  • I feel like everyone is forgetting that Remdesivir is only a viral inhibitor. Why do they expect it to magically treat severe cases when cytokine storms have infested the body? I firmly believe Remdesivir is effective, but it’s not THE cure. It should be paired with other drugs like immuno-repressants to stop cytokine storms in order to achieve maximum efficiency in treatment.

    • 100% correct. It’s an inhibitor. It inhibits the replication of the virus and it has demonstrated effectiveness in doing so against coronavirus since at least 2005 as noted in a NIH study (under Fauci!!!). It should be administered before the virus has had a chance to replicate to the point where it overwhelms the body. Naturally it’s inhibitive effects are not the solution once the virus has overwhelmed the body … but it can help prevent the virus from overwhelming the body if administered early enough. That is how the press ought to be framing this argument.

      Now … what motive does the press have to slant people against this potential life saving treatment?

  • I’m sure “X” would be the first in line to sign up in the study with the possibility of receiving a placebo. Oh and by the way ”X”would definitely insist that “X’s “ spouse enroll as well. The tragedy is that ”X’s spouse received the placebo and did not survive, while “X” received the actual drug and survived . And “X” is there alone holding a bag of “CHIPS”. Oh, and by the way the drug did work.

    • I am amazed that there are still those who insist on using the outdated “gold standard”. Today is a very different time where big data can be summoned to crunch over a million variables in a few minutes at most. With this power and such a large sample size, nearly every condition can be captured for comparison without the need for a placebo (death) arm to the study. Today more and more scientists are condemning this Las Vegas style trial method where the patient is asked to play Russian Roulette. It is one of the last shameful acts of science where death of a trusting human being is forsaken in the name of practicing what should be a forbidden practice.

Comments are closed.

Sign up to receive a free weekly opinions recap from our community of experts.
Privacy Policy