In a gigantic feat of scientific ambition, researchers have designed a staggering 1,200 clinical trials aimed at testing treatment and prevention strategies against Covid-19 since the start of January. But a new STAT analysis shows the effort has been marked by disorder and disorganization, with huge financial resources wasted.

The analysis, conducted in partnership with Applied XL, a Newlab Venture Studio company, found that one in every six trials was designed to study the malaria drugs hydroxychloroquine or chloroquine, which have been shown to have no benefit in hospitalized patients.

“If the goal was to optimize the likelihood of figuring out the best treatment options, the system is off course,” said Robert Califf, the head of clinical policy and strategy at Verily Life Sciences and Google Health and a former commissioner of the Food and Drug Administration.  The findings show, he said, that too often studies are too small to answer questions, lack real control groups, and put too much emphasis on a few potential treatments, as occurred with hydroxychloroquine.

advertisement

Indeed, the analysis found many of the studies are so small — 39% are enrolling or plan to enroll fewer than 100 patients — that they are unlikely to yield clear results. About 38% of the studies have not actually begun enrolling patients.

top drugs enrollment chart

“It’s a huge amount of wasted effort and wasted energy when actually a bit of coordination and collaboration could go a long way and answer a few questions,” said Martin Landray, a professor of medicine at Oxford University and one of the lead researchers on the RECOVERY study, a large trial of multiple treatments being run by the U.K. government.

advertisement

Not all effort has been for naught. Three of the most important conclusions about Covid-19 treatments so far have come from the RECOVERY trial. It has shown that dexamethasone, an inexpensive steroid, reduced the death rate of Covid-19 patients on ventilators by a third. It has also demonstrated that neither hydroxychloroquine nor a pair of HIV drugs, lopinavir and ritonavir — which had shown some early promise in laboratory models of the disease — benefit hospitalized patients with Covid-19.

Still, experts say the analysis shows that huge amounts of energy have been expended on haphazard efforts, often without a clear strategy to improve the odds that results would actually inform the care of patients. Faced with intense pressure to develop drugs and vaccines at previously unimaginable speed to push back a global pandemic, researchers may have actually slowed down the rate of progress.

For instance, 237,000 patient volunteers were to be enrolled in studies of hydroxychloroquine or chloroquine. That’s 35% of the 685,000 patient volunteers whom researchers hoped would be enrolled in any study. Since patients willing to enter studies are one of the scarcest resources in medicine, this means that other potential treatments, such as ivermectin or favipiravir, were not studied.

That’s “excessive,” said Susan Desmond-Hellmann, former CEO of the Bill and Melinda Gates Foundation. She noted that vaccines, in contrast, are being developed in a more methodical way — and wished that research on new drugs had been more organized, rather than simply trying whatever was available.

The data for the new analysis come from clinicaltrials.gov, the U.S. government database.

AppliedXL analyzed the data to identify trials that explicitly studied Covid-19 therapies and prevention strategies. The analysis focused only on interventional clinical trials, which study potential treatments or preventatives for disease. More than 880 observational studies have also been started, according to the analysis. For trials that test multiple treatments, the entire study enrollment was counted for each drug. (For more details on the methodology used in the analysis, see here.) 

The analysis reflects the data as of June 24. In the days before this article was published, approximately 20 Covid-19 studies were added to the database each day.

Studies started by month

The clinicaltrials.gov database is known to contain routine errors, oversights, and omissions, and researchers in both industry and academia often fail to update trial listings. As a result, the analysis itself involved some degree of subjectivity. But the general conclusions — both about the scale of the research and its limitations — seem indisputable.

Nahid Bhadelia, medical director of the Special Pathogens Unit at the Boston University School of Medicine, called the data “a cry for greater global collaboration during pandemics.”

America’s research infrastructure mobilized quickly when the pandemic began. In January, 10 studies were to be started, followed by 43 in February and 99 in March. In April, almost 400 studies for dozens of different treatments and preventatives were to begin, according to the database.

The sheer speed with which the studies were started was remarkable. And experts said the start of some small studies, particularly of new, experimental drugs that were previously being tested in other diseases, makes sense as a way of figuring out what might work. But such “phase 1” studies represented only 12% of the total in the analysis.

Experts added that, because the prognosis for patients with Covid-19 varies so dramatically — some patients have no symptoms, while others die on ventilators — only large studies that randomly assign patients to a treatment or placebo can deliver real insight into whether or not medicines are actually helping patients. Otherwise, researchers are fooled into thinking that differences between groups of patients with varying degrees of illness are caused by the medicines they are testing.

The RECOVERY study took a unique approach. In order to run such a large study, the researchers stripped down the amount of data collected on each patient — focusing mainly on whether patients lived or died — so that frontline researchers would be able to collect the data. More important, they got buy-in from the U.K.’s National Health Service that such a study was a priority. 

Repeating that model, experts agree, would teach doctors more about how to treat Covid-19, and do so much faster.

“If more people took the RECOVERY model, or something like it, did that for the drugs they were interested in, in the patients they’re interested in, in their part of the world … we make progress an awful lot quicker,” said Landray.

Clinical trials can routinely cost $10 million or more, with some studies costing hundreds of millions of dollars. But Landray says RECOVERY was funded by a grant of about $2.5 million to the center coordinating the study, although costs at hospitals could push the total higher.

In fact, of 1,200 studies, almost all the certain knowledge — and the proof that two treatments are effective — has come from two: RECOVERY and a study conducted by the U.S. National Institutes of Health that showed Gilead’s intravenous drug remdesivir speeds the time it takes for hospitalized patients to recover from Covid-19.

Data collected by AppliedXL and STAT show that researchers had little interest in studying dexamethasone, the only medicine shown to save the lives of Covid-19 patients. There have been seven studies of the drug in all, enrolling 13,600 patients, 12,000 of whom were in RECOVERY. Two other steroid drugs, prednisone and methylprednisolone, are being studied in another 2,500 patients.

status and purpose chart

The experience with hydroxychloroquine illustrates how the research enterprise became so lopsided and unfocused. In February, hydroxychloroquine was one of several medicines that showed promise in cell cultures as a potential antiviral treatment for the virus. Although none of these medicines was designed specifically to combat SARS-CoV-2, the coronavirus that causes Covid-19, there was real reason to hope one might work as an antiviral treatment to either help infected patients or prevent infection.

Early results from researchers in France ignited interest in the drug in March, even though they were not drawn from a randomized trial. On March 19, President Trump, at a press conference, said that hydroxychloroquine had “shown very encouraging — very, very encouraging early results,” and promised “we’re going to be able to make that drug available almost immediately.” The watchdog group Media Matters said that between March 23 and April 6, guests and hosts on Fox News mentioned the medicine nearly 300 times.

Huge donations of hydroxychloroquine and chloroquine were made by generic pharmaceutical manufacturers to a national stockpile of the drugs, leading the Food and Drug Administration to grant emergency use for its use in hospitals on March 28.

HCQ_and_chloroquine enrollment by purpose

By early April, 58% of patients hospitalized with Covid-19 were receiving the drug, triple the level in February, according to CarePort Health, which collects drug utilization data from electronic medical records. 

Instead of conducting studies that were large enough and well-designed enough to find out if hydroxychloroquine or chloroquine could prevent or treat Covid-19, many doses went to studies without control groups, essentially tracking the use of the medicine in hospitals. 

On Saturday, the World Health Organization said its own large study had found no benefit for either hydroxychloroquine or lopinavir-ritonavir. It’s still possible that one of the ongoing studies of hydroxychloroquine will show a benefit, perhaps earlier in the disease.

“The lack of leadership around a clinical trial agenda in the US is one of the failures of the US’s pandemic response,” said Walid Gellad, director of the Center for Pharmaceutical Policy and Prescribing at the University of Pittsburgh. “If we had taken the U.K.’s strategy of a set of large pragmatic trials, prioritizing recruitment in those trials, we could have all the answers now that we’re waiting for.” 

U.S. researchers rushed quickly into the battle against Covid-19. What was missing, it seems, was a general who could direct them in the fight.

Francesco Marconi contributed reporting. Joanna Lin Su contributed visuals.

  • Matt, this is a well done and intersecting story. Yes it does seem like F Troop sometimes. But I am not sure the drug research industry goes about finding cures for any specific disease targets in any kind of organized fashion. Probably should have in this case, but how would you go about it?

  • The chloroquine trials stink of corruption. Check the death rates for countries like uae South Korea etc

  • The front line critical care covid alliance of US physicians started using cortico steroids together with high dose IVC and heparin back in March to successfully treat covid patients. Their work has been ignored by politicians, academia and the mainstream media tied to the big pharma approach to medicine.
    Dr. Paul Marik, whose sepsis protocol has saved many lives, is one of the leading figures in the FLCC group of distinguished physicians. His work reveals that high dose IVC has powerful anti viral properties and has far more efficacy than billion dollar drugs such as Remdesivir. When will the media start publishing the truth about cheap safe medicines that can successfully treat covid 19?

  • This writer is clearly a shill for big pharma. I hope this writer burns in hellfire for eternity. But alas, as I don’t believe in hell, or heaven, I don’t believe this writer will get what this writer richly deserves. Shills like this should expect some punishment for facilitating the murderous calculations of profit driven medical entrepreneurs. The calculation is simple. You won’t pay big bucks for a vaccine unless people are dropping like flies. So anything that might stop that happening – like say HCQ and Zinc, must be denied. In many places this kind of propaganda had led to the banning of this cheap, and in many cases effective, option. So people like this writer have been facilitating mass murder, much in the same way that Goebels facilitated the Final Solution. I really hope there will be a Nuremberg at some point. People who propagandize for profit as the bodies are mounting up need to go to prison for a very very long time.

  • Ok we saw the viral load tank in humans in a small study with Raoult in March with a hydroxy/azith cocktail. then a 1000 person cohort with Raoult-mean age 43.5. SD I dont know , but he is a savvy scientist and knows what stacking the deck would be dishonest. Results?About 1% death rate-several times lower than US even. Now another study by Raoult 3700+ results? 0.9% death rate,mean age 45 with SD= 17. Come to America, Dr Zelenko-just released preprint- 80% reduction in death when compared to general death rate in America of about 5%. Some data unclear but he had many with comorbidities and 60+ age. Now Henry Ford study, 50% reduction in death rate with hydroxy.
    What do these have in common-the treatments were either initiated very early after positive diagnosis(the Henry Ford one initiating treatment latest). Some error due to perhaps some imperfect data with Raoult and Zelenko? Maybe. Enough to throw it off by a factor of 5? Nearly impossible. No my friends, there is something more than math and medicine at work here.

  • Innate immune system prevents the illness in early stage. It also reduce the symptoms. Several drugs that boost innate immune system can help not only to control covid-19 spread but also to prevent the suffering of patients. Most of the US researchers are just ignoring these facts and try to administer these drugs to hospitalized patients. Policy of admitting patients to hospital prevents the trials to be conducted with the persons who can get benefit from these drugs. It is very unfortunate poor patients are suffering because of wrong procedures adopted by so called health experts who cannot understand the basics. Unfortunately US president also seems to be powerless to avoid this injustice to the patients as most of the Media and the other forces are blind to the actual situation, while several other countries such as Japan are getting benefit by utilizing of the appropriate procedures

  • Underlying this piece are certain assumptions about research design, making the case once again for targeted research that is sufficiently powered. The opposing case needs to be argued with equal vigor. We are in a complex variable space, and we are fighting exponential growth on three fronts: spread of the vector in the population; viral growth in the affected individual; and exponential growth of the counter-measures — the cytokine storm.

    All these call for early interventions, before each of these problems becomes unmanageable. Two of them fall in the realm of public health—the realm of prevention—and only one of them is in the class of medical remedies that should be subject to tightly specified research designs.

    For the other two, complexity is inherent in the situation. It must be confronted forthrightly, and it should be met with numerous independent small studies, just as is presently happening. The results are then distilled by way of data-mining across studies. Whether formally or informally, with respect to prevention strategies we find ourselves in a Bayesian universe.

    We should be focusing on prevention with almost paranoid intensity. This is where hydroxychloroquine fits (in combination with zinc), as well as ivermectin, each in combination with an antibiotic to ward off opportunistic infections. This is the place for bottom-up scientific discovery and early sifting through clinical pipelines.

    • Why does hydroxychloroquine fit into a preventative strategy better than a vaccine (genuinely curious, not trying to argue)?

      It seems like the more prevantative a solution becomes, the larger the study needs to be to have correct power and prove benefits outweigh risks. This would seem to suggest larger, more expensive studies and good coordination would be needed for that.

      Also, I keep seeing people criticizing studies for not focusing on early treatments, where is the evidence that this is the correct approach?

  • Yes, coordination would be better. However, there is money to be made in some instances, so the idea that national and commercial interests will not be a factor is not what happens in the real world. Also, randomization that is not based on known stages of the disease will arrive at the wrong conclusions. Stat keeps making this error too, just as the trials can.

    • Response to Happy Cow: HCQ and ivermectin are ‘bird-in-hand’, so that is better than a non-existent vaccine. We’ll have a dynamic situation for some time to come. As for preventive solutions calling for even larger studies, several responses: First of all, these old drugs are already well-known quantities in other contexts. Secondly, patients tend to respond quickly, so the clinician can discern whether the drug is actually helpful for a particular patient, and also whether the dose needs to be adjusted. Statistics of responding are then accumulated over time. Further, the criticality of timing and the diversity of clinical presentations makes this situation totally unsuitable for classical research designs.

      Where is the evidence that this strategy is actually being successful? This is how the death rate has been reduced in ICUs, for example. It did not happen because of any research. It happened by clinicians sharing their insights with each other, one clinical anecdote at a time. Trial-and-error learning is the appropriate response to the encounter with a complex system. Formal research designs cannot function on the timescale at which we actually have to solve our current problems.

      The large-scale outcome study in France put HCQ plus zinc plus Azithromycin firmly on the map. The results cannot be explained away with arguments about lack of controls with random assignment. But that also does not mean we have nothing better… the exploratory research continues.

      Prevention strategies are inherently handicapped when it comes to proof. Consider, for example, the Covid-19 disaster that is not happening in most of Africa. That could be for a number of reasons, but it could also be because chloroquine is already well established there.

Comments are closed.

Sign up to receive a free weekly opinions recap from our community of experts.
Privacy Policy