The world wants answers on Gilead’s Covid-19 drug. Experts worry next studies may increase uncertainty

For weeks, the world has been eagerly awaiting clinical trial results for one experimental drug, remdesivir, to treat Covid-19. On some days, the entire stock market has moved up and down based on limited amounts of data about the therapy from Gilead Sciences.

The signals, so far, have been contradictory. An early peek at one study, based on data from patients treated at a Chicago hospital, suggested patients were doing better than expected on remdesivir. Days later, a summary of results from a study in China showed that patients on the drug did not improve more than those in a control group. 

A fuller picture on remdesivir is expected any day now. Gilead has said that it will release data from one of its U.S. trials — the one from which the Chicago results were disclosed — by the end of this week.  Even more data, from other trials including the China study, could follow shortly thereafter.

But outside experts in clinical trial design worry that the results, instead of leading to a clear picture of whether the medicine is effective, will instead muddy the waters further.

The main concern, they say, stems from the fact that the Gilead trial expected to read out this week, which was conducted among patients with severe disease, lacks a control group — that is, patients who are randomly assigned to receive the best treatment available, but not remdesivir. As designed, the only randomization is the duration of treatment: either five days or 10 days of drug. Without a true control group of patients, many experts say, it will be difficult to determine whether remdesivir is effective.

“The overall study itself has little or no scientific value since all patients are receiving the drug,” said Steven Nissen, the chief academic officer at the Cleveland Clinic and lead investigator of many trials for heart drugs that have been approved by the Food and Drug Administration.

“The study, as designed, is essentially useless and cannot be used by the FDA for consideration of remdesivir for approval to treat coronavirus,” Nissen said.

Peter Bach, director of the Center for Health Policy and Outcomes at Memorial Sloan Kettering Cancer Center, called the situation “frustrating.”

“For them to run the trial in severe but not include a control group, it’s just such a waste,” Bach said.

The predicament is symptomatic of one of the biggest problems in medicine: Collecting data quickly can actually slow things down if studies are not designed in a way that gives clear, definitive answers.

Not everyone is as grim about prospects for useful data. Scott Gottlieb, the former commissioner of the FDA and a fellow at the American Enterprise Institute, defended the Gilead study. He noted that while the trial expected to read out first is “open label”— meaning that both doctors and patients know who is getting the drug and who is not — more trials are still being conducted.

“There’s a rigorous, placebo-controlled trial that’s now fully enrolled and will read out soon, but there’s also a place for open label studies to broaden the safety database and answer discrete questions,” Gottlieb said, referring to a study sponsored by the National Institute for Allergy and Infectious Diseases.

Gilead says it had ‘only limited supply’

Gilead said that its study was never meant to be the first word on remdesivir’s efficacy. Instead, it was meant to follow two studies from China and the U.S. government study, run by NIAID.

“In the early stages of the pandemic, we not only had a limited supply of remdesivir but also a limited supply of the matched placebo required for placebo-controlled studies,” said Amy Flood, a Gilead spokesperson. “We chose to prioritize manufacturing active drug over placebo, and we provided our supply of placebo to China and NIAID for their studies of remdesivir.”

Given that three randomized, placebo-controlled trials were either underway or about to start, Flood said, Gilead designed its own studies to be open label. But this makes any conclusion from a study far weaker, because unconscious biases can affect the results.

Critics point to Gilead’s decision to compare two groups given remdesivir for either five days or 10 days. The problem with this strategy, they say, is that an ineffective drug that did nothing and a very effective drug that consistently helped patients overcome the virus would look the same in such a study. Only if the 10-day course were more effective, or if it was worse because of side effects, would the study have any clear result.

Gottlieb defended Gilead, arguing that increasing the size of the study from 400 patients to 6,000 expanded access to the drug at a time when it was needed to help with the public health emergency, but also allows the company to collect more rigorous data. “I think it’s far better to provide access in an open label setting than to merely give drug away in an expanded access protocol where you’re collecting minimal or no data,” he said.

Yet another trial in less sick patients, also run by Gilead, does have a control group and may give a clearer answer. Nissen sees “a reasonable study design.” But Bach was more critical, saying that even though that study has a control group, the lack of a placebo means the study might not be trustworthy. That’s because its main goal, time to improvement of symptoms, could be affected by the perceptions of clinicians and the patients themselves.

Bach said the hospitals conducting the study “are easily capable of wrapping syringes in brown paper and blinding the whole thing. I don’t understand why you would run a trial like this.”

The problem of placebo

When case numbers in the U.S. started to spike in mid-March, Northwell Health, New York’s largest health care provider, immediately staffed a 200-person unit to run Covid-19 clinical trials. Kevin Tracey, Northwell’s executive vice president of research, said that the group’s goal was to conduct rigorous clinical trials that included control groups. Why not have a placebo arm in the Gilead-run study of severe patients? 

“Obviously, that’s my first choice,” Tracey said. One reason it didn’t happen was that everyone needed to move very fast.

“In a quiet time when you have a few years to figure out the best way to lower cholesterol or the best way to treat arthritis, there’s a dialogue between the investigators and the company and the FDA and you negotiate the trial design,” Tracey said. “We didn’t do that with Gilead, in this case, because our hospitals were filling up with people who were dying.”

But Tracey points to another problem, too. He said that more severely affected patients and their families, having read optimistic news reports about a medicine, may not agree to be randomly assigned to placebo.

“That’s a reality,” he said. “You know that your mother, your wife, or your father, your brother is in the bed dying and you’re supposed to sign the papers that say you might get a placebo? No thank you, a lot of people say.”

At the same time, it can be hard to answer questions without randomized data, and even then results can be confusing.

What to expect from Gilead’s study

Gilead’s severe Covid-19 study is enrolling approximately 6,000 participants from 152 different clinical trial sites all over the world. The primary analysis to be disclosed this week is expected to encompass data from at least 400 patients, with a statistical comparison of patient improvement between the two remdesivir treatment arms — the five-day and 10-day treatment course. Improvement will be measured using a seven-point numerical scale that encompasses death (at worst) and discharge from hospital (best outcome), with various degrees of supplemental oxygen and intubation in between.

Video footage of a town hall at the University of Chicago, seen by STAT, provides some clues about what the results might bring. The researcher speaking in the video, Kathleen Mullane from the University of Chicago, said, “Most of our patients are severe and most of them are leaving at six days, so that tells us duration of therapy doesn’t have to be 10 days. We have very few that went out 10 days, maybe three.”

If Chicago’s experience holds true at the other hospitals participating in Gilead’s study, there will be little or no difference in patient improvement between the two remdesivir arms, making the comparison uninterpretable.

That, though, will not be the end of the conversation or the debate. 

Gilead could also provide additional analyses on the time to clinical improvement for different subsets of remdesivir patients, based on disease severity, onset of symptoms, or how quickly treatment was started. Without a control arm, these data will be hard to interpret on their own. 

One approach: compare to other trials, like the 199-patient study of the HIV drug lopinavir that was published in the New England Journal of Medicine in March. In the study conducted in China, lopinavir was 31% better than standard of care in time to clinical improvement, but the result was not statistically significant. Mortality was similar, with 19% of lopinavir-treated patients versus 25% of standard of care patients dead after one month.

Baird analyst Brian Skorney, for instance, points out that 79% of the lopinavir-treated patients were cured at one month, yet there was still no significant difference from the 70% cure rate in the control arm. This suggests that high rates of patient recovery should be expected, and that remdesivir — a modestly effective antiviral at best — will not make a big difference.

But RBC Capital biotech analyst Brian Abrahams suggests using the control arm from the lopinavir trials as an admittedly rough comparator when the Gilead data are released.

At seven days, 2% of the patients treated with standard of care showed a clinical improvement, increasing to 30% at day 14 and 70% at day 28. If remdesivir improves upon those results, perhaps doubling the time to clinical improvement at day 14, that might be seen as suggesting a benefit, Abrahams believes. 

Umer Raffat, an analyst at Evercore ISI, pushes back against the conclusion that lopinavir failed in the China study — which might bode well for remdesivir. Raffat points to an analysis in the NEJM study that showed in a small group of less sick patients treated before their symptoms worsened significantly, the lopinavir mortality rate was 45% lower.

Antiviral medicines should be more effective when given earlier. This is what Gilead is trying to do with remdesivir in its severe Covid-19 study. The company is expected to report mortality rates when it announces results later this week. If remdesivir lowers mortality to 10-15% from the mid-20% range reported in other studies, that might also suggest a benefit. 

Comparisons across different clinical trials may help paint a clearer picture of remdesivir’s efficacy, but they’re not likely to persuade FDA to approve the drug. That will only come from positive data in subsequent clinical trials. 

What to expect: the China-run severe study

The trial in severe patients conducted in China — the one accidentally posted on the WHO’s website last week — failed to show that remdesivir sped the time it took for patients to improve. After one month, it appeared 13.9% of the remdesivir patients had died compared to 12.8% of patients in the control arm. The difference was not statistically significant.

But the WHO did not post full data; rather, it was an abstract of the study and published without the full manuscript. It’s possible that the published version will be different.

It’s even possible this study will be supportive of some effect for remdesivir. The key is a statistic, known as a hazard ratio, showing that the time to clinical improvement for patients in one group was 23% more than the other. The key question is whether patients did better on remdesivir or placebo.

Raffat, the Evercore ISI analyst, obtained a copy of the study protocol, a plan for how it was going to be run, and wrote in a note to the bank’s clients that this figure was planned to be 1.4 if remdesivir were to have a statistically significant benefit. If that holds true, the 1.23 figure means that the study just missed showing the drug was effective, leaving hope that future studies could show the drug would be effective. 

“It’s not a silver bullet, but I do believe remdesivir is active,” Raffat said on a podcast for investors Friday.

The most clarity will come from the study from NIAID. It is controlled with a placebo, and patients and doctors are blinded to treatment. Full results are not expected until the end of May, but the study’s lead investigator, Andre Kalil, told Reuters on Friday that an interim look at the data may be available in a week or two.