When Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases, described the results of a remdesivir study a couple of weeks ago, he was cautious in characterizing how the experimental Gilead Sciences drug helped combat Covid-19.
Patients given the intravenous medicine recovered faster than those on a placebo by 31%, or four days, Fauci said, conceding the result was not a “knockout.” Nonetheless, he insisted in his trademark keep-it-simple-demeanor, that the data showed “remdesivir has a clear-cut, significant, positive effect…. This has proven that a drug can block this virus.”
By and large, this was good news, but there was a caveat — the data were based only on a preliminary analysis. Rather than release all the data, the National Institutes of Health, which sponsored the study, issued a press release saying a forthcoming report would have “more comprehensive data.”
The rationale is easy to understand. In a pandemic, who doesn’t want insight into a useful therapy as soon as possible? So on this basis, the data were sufficient to allow the Food and Drug Administration to quickly authorize emergency use in hospitals.
Even so, this is not the best approach to science or medicine.
Rather than release information in dribs and drabs, study sponsors — whether a government, company, or university — should release all the results lickety-split, especially when it comes to medicines for fighting a pandemic. Data should not be held back.
“If we can’t see the study protocol and a concise research report, you shouldn’t be talking about it in the news,” said Joseph Ross, a professor medicine and public health at Yale University, who studies clinical trial practices. “Instead, we now have a situation where partial data is disclosed and physicians have to use that to make a decision in the moment.”
Indeed. In the absence of full results, we lack further insight into safety or how the drug may work on subsets of patients. And for now, the extent to which remdesivir can prevent deaths – a key metric – is unsettled. Fewer patients died while on the drug than placebo, but the findings were not statistically significant and need further analysis. Fauci, however, insisted “the conclusion will not change.”
Maybe not, but until the data are released, doctors are left with a mix of facts and assumptions.
“When we look below the hood, we may find other issues,” said Lewis Nelson, who chairs the Department of Emergency Medicine at Rutgers New Jersey Medical School and University Hospital in Newark. “We’ll eventually get the info, but why shouldn’t we get it now? Whether it’s two weeks or two months, we’re making decisions based on what they’ve given us. But we run the risk of making a decision based on the wrong info.”
David Hill, a pulmonologist affiliated with two hospitals in Waterbury, Conn., agreed. “While decreasing [the] length of stay [in a hospital] is positive, decreasing illness and severity of illness is more important,” he told us. “At this point, I view it as a possibly beneficial therapy in critically ill patients. I would be hesitant to embrace it wholeheartedly until the full trial results were available to review. Without those results, I am more skeptical.”
Of course, no one is suggesting that remdesivir should not be given to Covid-19 patients, despite the incomplete picture. In general, though, the sooner trial data are available for review and scrutiny, the faster physicians and researchers can gain a clearer picture into the best use of a medicine and also understand when it should not be used.
For his part, H. Clifford Lane, the clinical director at the NIH division that sponsored the study, told us that “the full results have been submitted for peer review by a journal. We anticipate a quick review.”
Unfortunately, there is no mandate for reporting results immediately. In fact, a federal law called the Food and Drug Administration Amendments Act requires institutions sponsoring clinical trials to post results to a U.S. government website called ClinicalTrials.gov one year after a study is completed. Sometimes, this can be extended to three years.
But who has time to wait for study results during a pandemic?
Meanwhile, an increasingly popular avenue for disclosure has become the pre-print server, an online forum where researchers can post studies that have not yet been reviewed or published. Last June, one popular site called MedRxiv posted fewer than 100 of these working papers; last month it posted more than 1,500.
However, as the site noted, these papers “should not be relied on to guide clinical practice.” Why? A paper that is not vetted may report selective outcomes, underreport side effects or reach inappropriate conclusions. In short, there is a lack of quality control, according to Deborah Zarin, a former director of ClinicalTrials.gov, who is now program director at the Multi-Regional Clinical Trials Center that is run by Brigham and Women’s Hospital and Harvard University.
By contrast, ClinicalTrials.gov entries are guaranteed to address minimum information needed to interpret studies and the emphasis is on summary data, with minimal opportunities for narrative spin, Zarin explained. Moreover, the data undergo rigorous quality control so that entries are meaningful and internally consistent. And the data must be consistent with the pre-specified study protocol.
“The research community could demonstrate its commitment to serving the public interest by submitting the results of Covid-19 trials to ClinicalTrials.gov as soon as they are available,” she said. “The medical community should not have to sit around in the dark, while critical policy and clinical decisions are being made based on conclusions, but no data, that are promulgated by researchers or sponsors.”
Not everyone is so accommodating. There is, however, a provision in federal law to make that happen. Specifically, the NIH director can require trial results to be posted in 30 days after it has been determined that doing so would be in the interest of public health.
“It doesn’t require immediate posting, but it’s certainly one avenue the federal government can use to disseminate [results of] a particular clinical study,” said David Clissold, an attorney at the Hyman, Phelps & McNamara law firm, which specializes in FDA and regulatory matters.
In this instance, however, the NIH sponsored the remdesivir study. So maybe a different kind of push is needed. Ross has a suggestion — and it makes sense. Before the FDA issues an emergency authorization for a Covid-19 treatment, the trial results should be fully and publicly disclosed. “The FDA has the best lever that could be pulled,” he said. “Otherwise, it’s a ‘trust us’ decision.”
At the end of the day, the data — all of the data — should speak for themselves. Science can be messy enough without incomplete pictures being disseminated during a public health crisis.
Good observations by Ed Silverman on transparancy that is crucial for proper drug use, and that is being withheld, or incurring unacceptable delay. This reeks of NIH corruption, such as personal interests in Gilead stocks. There is something fishy about the hyped-up ramp-up of an at best only mediocre drug. The FDA should be as firm with Gilead as it is with the Bluebird Bio / Bristol Meyers new drug application. If the FDA is not, then who is in bed with who – and how many are there under the sheets?
Please note that the preliminary results were released by the sponsor, not the investigators. The decision to do so was based on a Data & Safety Monitoring Board (DSMB) review. The interim analyses done for a DSMB are not as complete or thorough as one would want for publication, plus a final report done after the DSMB review will have more recent data.
I’d guess the investigators want to get the analyses out as soon as they can do so responsibly, with a report that examines whether certain types of patients are more likely to respond, thoroughly lays out safety data, etc. To do that properly takes a few weeks even if the statisticians are doing analyses at full speed – the analyses have to be reviewed, the investigators need to write up the results, and they don’t want to rush so much that analysis mistakes are made or results misinterpreted.
But the data was sufficient for FDA to review & act on.
That means the data is sufficient for publication or at least posting online.
This is a huge trial with the only approval. A journal will still allow it (overlooking their own policies on pre-publication).
@JC – in normal times that would be true. But currently the FDA is approving things related to COVID-19 with much less data than they’d usually require. As a statistician who works in clinical trials and has prepared/presented to DSMBs, I can assure you that the report for a DSMB review isn’t nearly thorough enough to get FDA approval of anything. The statistician is currenly doing more analyses, having them checked internally, making tables and figures and helping the investigators write the draft paper.
The role of a DSMB is general oversight. When one stops a study early, it’s because it doesn’t consider continuation ethical for reasons such as one arm being clearly much better than the other and it’s unethical to continue submitting participants to the inferior arm. That doesn’t require the same amount of data as an FDA approval, which is normally a long slog.
If the study team were to have released a preprint right after the DSMB, it would likely have contained the same info that’s already been shared plus some preliminary side-effects data that’s probably similar to what’s been seen in the other remdesivir studies to date. If there’s still nothing out in a couple more weeks, it will be reasonable to complain. But right now the time taken seems very reasonable to a person who has actually worked on clinical trials.
Statistical significance is an arbitrary and misleading threshold, it’s better to state the p-value and the model.
General public first heard of new corona virus kin about mid January and now generally agreed that there were cases in November 2019. Many of the symptoms are mild and never reported to any doctor or medical facility so we cannot be sure of where or when the new virus originated. Wuhan is a commercial, industrial and air transportation hub. China’s Belt and Road Initiative resulted in wide spread interactions between China, Central Asia, Middle-East, Europe and US. There was a major “Fashion Week” in Milan just before the outbreak in northern Italy and Chinese fashion enterprises had connections with the Milan fashion industry and there was frequent communication between Iran and Wuhan. It is plausible, if not probable, that many asymptomatic people with the new virus were widely dispersed throughout the world and some who were diagnosed with pneumonia had Covid 19. This makes the available data incomplete and possibly misleading. Early on a number of statisticians recommended testing or reviewing a random population sample of world populations to make an informed estimate of where, when and how many may be infected. I am not aware of such an effort. Certainly selecting the sampled population is difficult, but it should be done if it has not been done. The death rates for the new virus should be compared with the death rates before the virus appeared. Data is reported that is clearly in error with no comment, and what is reported, often seems selected to support hypotheses. Political biases are clear.
May one cynically observe that since it seems to be so difficult to get this drug / infusion of the moment distributed, do physicians need to decide how to administer something they do not have available? That said, I would prefer that they did have it – along with some treatment protocols however preliminary.
Excellent point and I feel strongly that all the Senators missed the golden opportunity to ask Anthony Fauci about it yesterday when he was answering many other questiond. I think he would have disclosed something and the timing. It was stupid of them IMO!
Almost like the Barr summary of the Mueller Report, deja vu again. How difficult to be transparent unless there is a need to hide?
Here what I wrote before.
May 5, 2020 at 7:50 pm
Hidden results! If it is such a clear benefit why can’t we see all the data (like CPR results – positive to negative conversion, patient characteristics, oxygen saturation levels, other interventions, heterogeneity in outcome between different hospitals, etc). Considering the Hubei trial came out negative, as usual with Gilead, once again they are in murky grounds including their own strange results from 5d v. 10d trial (is 1d as good as 5d?) and their rhesus model (why not include interventions at 12h, 24h, 48h after the inoculation challenge?).
Were researchers wrong to move the goalposts on remdesivir? In the end, it may not have mattered
By Matthew Herper @matthewherper
May 5, 2020
Hi E. Cantekin,
Thanks for your note.
This is not just about Gilead and remdesivir, though, or any one study. As more trials test more products for Covid-19, the issue is the need for complete results as soon as possible, in general.
ed at pharmalot
Comments are closed.