Skip to Main Content

The current coronavirus disease, Covid-19, has been called a once-in-a-century pandemic. But it may also be a once-in-a-century evidence fiasco.

At a time when everyone needs better information, from disease modelers and governments to people quarantined or just social distancing, we lack reliable evidence on how many people have been infected with SARS-CoV-2 or who continue to become infected. Better information is needed to guide decisions and actions of monumental significance and to monitor their impact.

Draconian countermeasures have been adopted in many countries. If the pandemic dissipates — either on its own or because of these measures — short-term extreme social distancing and lockdowns may be bearable. How long, though, should measures like these be continued if the pandemic churns across the globe unabated? How can policymakers tell if they are doing more good than harm?

advertisement

Vaccines or affordable treatments take many months (or even years) to develop and test properly. Given such timelines, the consequences of long-term lockdowns are entirely unknown.

The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable. Given the limited testing to date, some deaths and probably the vast majority of infections due to SARS-CoV-2 are being missed. We don’t know if we are failing to capture infections by a factor of three or 300. Three months after the outbreak emerged, most countries, including the U.S., lack the ability to test a large number of people and no countries have reliable data on the prevalence of the virus in a representative random sample of the general population.

advertisement

This evidence fiasco creates tremendous uncertainty about the risk of dying from Covid-19. Reported case fatality rates, like the official 3.4% rate from the World Health Organization, cause horror — and are meaningless. Patients who have been tested for SARS-CoV-2 are disproportionately those with severe symptoms and bad outcomes. As most health systems have limited testing capacity, selection bias may even worsen in the near future.

The one situation where an entire, closed population was tested was the Diamond Princess cruise ship and its quarantine passengers. The case fatality rate there was 1.0%, but this was a largely elderly population, in which the death rate from Covid-19 is much higher.

Projecting the Diamond Princess mortality rate onto the age structure of the U.S. population, the death rate among people infected with Covid-19 would be 0.125%. But since this estimate is based on extremely thin data — there were just seven deaths among the 700 infected passengers and crew — the real death rate could stretch from five times lower (0.025%) to five times higher (0.625%). It is also possible that some of the passengers who were infected might die later, and that tourists may have different frequencies of chronic diseases — a risk factor for worse outcomes with SARS-CoV-2 infection — than the general population. Adding these extra sources of uncertainty, reasonable estimates for the case fatality ratio in the general U.S. population vary from 0.05% to 1%.

STAT Reports: STAT’s guide to interpreting clinical trial results

That huge range markedly affects how severe the pandemic is and what should be done. A population-wide case fatality rate of 0.05% is lower than seasonal influenza. If that is the true rate, locking down the world with potentially tremendous social and financial consequences may be totally irrational. It’s like an elephant being attacked by a house cat. Frustrated and trying to avoid the cat, the elephant accidentally jumps off a cliff and dies.

Could the Covid-19 case fatality rate be that low? No, some say, pointing to the high rate in elderly people. However, even some so-called mild or common-cold-type coronaviruses that have been known for decades can have case fatality rates as high as 8% when they infect elderly people in nursing homes. In fact, such “mild” coronaviruses infect tens of millions of people every year, and account for 3% to 11% of those hospitalized in the U.S. with lower respiratory infections each winter.

These “mild” coronaviruses may be implicated in several thousands of deaths every year worldwide, though the vast majority of them are not documented with precise testing. Instead, they are lost as noise among 60 million deaths from various causes every year.

Although successful surveillance systems have long existed for influenza, the disease is confirmed by a laboratory in a tiny minority of cases. In the U.S., for example, so far this season 1,073,976 specimens have been tested and 222,552 (20.7%) have tested positive for influenza. In the same period, the estimated number of influenza-like illnesses is between 36,000,000 and 51,000,000, with an estimated 22,000 to 55,000 flu deaths.

Note the uncertainty about influenza-like illness deaths: a 2.5-fold range, corresponding to tens of thousands of deaths. Every year, some of these deaths are due to influenza and some to other viruses, like common-cold coronaviruses.

In an autopsy series that tested for respiratory viruses in specimens from 57 elderly persons who died during the 2016 to 2017 influenza season, influenza viruses were detected in 18% of the specimens, while any kind of respiratory virus was found in 47%. In some people who die from viral respiratory pathogens, more than one virus is found upon autopsy and bacteria are often superimposed. A positive test for coronavirus does not mean necessarily that this virus is always primarily responsible for a patient’s demise.

If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths. This sounds like a huge number, but it is buried within the noise of the estimate of deaths from “influenza-like illness.” If we had not known about a new virus out there, and had not checked individuals with PCR tests, the number of total deaths due to “influenza-like illness” would not seem unusual this year. At most, we might have casually noted that flu this season seems to be a bit worse than average. The media coverage would have been less than for an NBA game between the two most indifferent teams.

Some worry that the 68 deaths from Covid-19 in the U.S. as of March 16 will increase exponentially to 680, 6,800, 68,000, 680,000 … along with similar catastrophic patterns around the globe. Is that a realistic scenario, or bad science fiction? How can we tell at what point such a curve might stop?

The most valuable piece of information for answering those questions would be to know the current prevalence of the infection in a random sample of a population and to repeat this exercise at regular time intervals to estimate the incidence of new infections. Sadly, that’s information we don’t have.

In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work. School closures, for example, may reduce transmission rates. But they may also backfire if children socialize anyhow, if school closure leads children to spend more time with susceptible elderly family members, if children at home disrupt their parents ability to work, and more. School closures may also diminish the chances of developing herd immunity in an age group that is spared serious disease.

This has been the perspective behind the different stance of the United Kingdom keeping schools open, at least until as I write this. In the absence of data on the real course of the epidemic, we don’t know whether this perspective was brilliant or catastrophic.

Flattening the curve to avoid overwhelming the health system is conceptually sound — in theory. A visual that has become viral in media and social media shows how flattening the curve reduces the volume of the epidemic that is above the threshold of what the health system can handle at any moment.

Yet if the health system does become overwhelmed, the majority of the extra deaths may not be due to coronavirus but to other common diseases and conditions such as heart attacks, strokes, trauma, bleeding, and the like that are not adequately treated. If the level of the epidemic does overwhelm the health system and extreme measures have only modest effectiveness, then flattening the curve may make things worse: Instead of being overwhelmed during a short, acute phase, the health system will remain overwhelmed for a more protracted period. That’s another reason we need data about the exact level of the epidemic activity.

One of the bottom lines is that we don’t know how long social distancing measures and lockdowns can be maintained without major consequences to the economy, society, and mental health. Unpredictable evolutions may ensue, including financial crisis, unrest, civil strife, war, and a meltdown of the social fabric. At a minimum, we need unbiased prevalence and incidence data for the evolving infectious load to guide decision-making.

In the most pessimistic scenario, which I do not espouse, if the new coronavirus infects 60% of the global population and 1% of the infected people die, that will translate into more than 40 million deaths globally, matching the 1918 influenza pandemic.

The vast majority of this hecatomb would be people with limited life expectancies. That’s in contrast to 1918, when many young people died.

One can only hope that, much like in 1918, life will continue. Conversely, with lockdowns of months, if not years, life largely stops, short-term and long-term consequences are entirely unknown, and billions, not just millions, of lives may be eventually at stake.

If we decide to jump off the cliff, we need some data to inform us about the rationale of such an action and the chances of landing somewhere safe.

John P.A. Ioannidis is professor of medicine and professor of epidemiology and population health, as well as professor by courtesy of biomedical data science at Stanford University School of Medicine, professor by courtesy of statistics at Stanford University School of Humanities and Sciences, and co-director of the Meta-Research Innovation Center at Stanford (METRICS) at Stanford University.

  • The author states “If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths.

    Diamond Princess allows the author to make assumptions to support his hypothesis and I certainly hope he is right and everyone else is wrong.

    However if his assumption is wrong and instead of infecting 1% of the population, coronavirus afflicts 10% of the population, the mortality would be 100,000 and the hospitals will be overwhelmed by the epidemic (20% necessitate admission and it will translate in 2,000,000 of inpatients).

    I am afraid that we need to see what is happening in the world, the hospitals in France, Italy and Spain are overwhelmed, and base our decision on the evidence we have, not the one we would like to have

  • Finally the voice of reason! I’ve been telling people this all along. Looking at the best data I can find coming out of South Korea where so many people have been tested the death rate is .77 of a percent. That is less than seasonal influenza! And the majority of those people are the elderly over 70 years of age where the death rate skyrockets. But that is almost always the case with infection in the elderly.
    In my opinion this whole thing is being blowing severely out of proportion and I mean severely! I think more damage will be done and more deaths will occur from the economic Fallout of this lunacy. This is nothing more than manufactured Mass Hysteria! What angers me the most are the so-called experts who are freaking everyone out without any amount of reasonable data backing up their Preposterous claims. I fell for it myself until a New York Doctor Who wrote a book about pandemics said that he was going by the only accurate statistics that he knew which came out of South Korea so I vetted this and found out it’s absolutely true! The numbers are being blown way out of proportion to the point of lunacy! If you don’t believe me or him do the research yourself it’s out there. Time Magazine wrote an article alluding to this several days ago. Look at the data yourself and make your own conclusions and don’t believe all the craziness that’s being spouted by the so-called experts who are not basing their conclusions on reasonable data but are rather blowing this way out of proportion without facts and claiming they know when they don’t simply because they don’t have the facts!
    The vast majority of people who are not elderly or have severe underlying medical disorders will be just fine. Please note that the higher proportion of elderly in a country like China or Italy will increase those numbers you must Factor this into your research. In the end I think we will find out that the economic damage will cause far more damage and loss of life than this virus. I hope this helps!

  • The new data, just out, from Italy, would indicate that we are dealing with a relatively low grade pathogen. Except for two individuals, all fatalities have been in the 80-89 age group, and all but two of this group had pre-existing serious medical problems. Only two people of that cohort were not implicated with a prior existing condition.
    Two deceased were reported younger than the 80-89 group. These two also had prior existing serious illness.
    It seems that the whole response is a terrible waste of money and inconvenience.

    • The age of those have died in Italy isn’t as dramatically weighted toward those over 80 as J.A. Pate states, but fatalities definitely are concentrated among the elderly.

      Per an Italian study as reported by Bloomberg News – https://www.bloomberg.com/news/articles/2020-03-18/99-of-those-who-died-from-virus-had-other-illness-italy-says

      “The average age of those who’ve died from the virus in Italy is 79.5. As of March 17, 17 people under 50 had died from the disease. All of Italy’s victims under 40 have been males with serious existing medical conditions.”

      “The Rome-based institute has examined medical records of about 18% of the country’s coronavirus fatalities, finding that just three victims, or 0.8% of [that] total, had no previous pathology. Almost half of the victims suffered from at least three prior illnesses and about a fourth had either one or two previous conditions.”

      Eyeballing a bar chart in the Bloomberg article, here’s the distribution of fatalities in Italy by age range:

      90+: ~200
      80-89: ~800
      70-79: ~650
      60-69: ~200
      50-59: ~75
      Under 50: 17

  • I completely agree that we should do try to implement randomly sampled testing asap to improve models. However, there is already enough data from Wuhan and Italy about what happens without sophisticated contact tracing/extensive testing (South Korea) or strict lockdowns (late stage Wuhan). While the true mortality rate is unknown, it seems that the fact that a large part of the population has no immunity and that COVID19 is much more contagious than the flu is enough to overwhelm health care systems. We need to act fast to flatten the curve to give us enough time to get better data and figure out a long-term plan. If healthcare systems are overwhelmed due to large numbers of people coming down with COVID19 at once (which again doesn’t happen with the flu because of vaccines, immunity and lower transmissability), it doesn’t matter if the original mortality rate would have been similar to the flu – it will go up significantly.

  • I am so glad Dr. Ioannidis wrote this measured article. Thank you for using the Diamond Princess data, as that makes the most sense of what to apply to the US population. As a phd student in the school of public health, I would also like to bring up some other discrepancies I see with the current modeling that may be worthy of note.If I am wrong, please correct me because I am a student and I am still learning.

    1. A lot of the models are using data that has a different distribution to what we would actually see in the US as the modeling is based on the more severe strand (L), even though that strand is becoming less prevalent globally.The L strand is predominantly in Asia and specifically in Wuhan.This is skewing the results to a more severe CFR and general outcome. Modeling based off the S strand would be more helpful.

    2. Although preliminary, a pre-print paper was just released suggesting temperature and humidity significantly decrease rate of transmission. Are any of the models including this as a covariate?

    3. 87 percent of the US population is under 65 compared with Italys 23 percent of the population over 65. Extrapolating an Italian like scenario doesn’t make sense acknowledging that fact alone (I could be wrong).

    4. The economic consequences of the media hysteria may translate into poorer public health outcomes if the economy is thrown into a recession .

    • I’d love to see the data about the different strands; I can’t find anything recent about their relative severities. Also what I’m seeing is that the L-type is more prevalent, the S being older.

  • I think some of you are missing his point. The point isn’t that there is a possibility that this is minor, it’s that our data are so bad that we cannot come close to apportioning uncertainty. It could be a bad flu disease (seems unlikely based on observed outcomes in Wuhan and Italy), but it could be a 1% mortality incident with even more extreme outcomes to the economy. We don’t know which is likely or how much our interventions will change any of this.

    The solution isn’t to just pretend the best case is the most likely, or that the one good data set we have (Princess Cruise) is representative of the world at large (especially when we have empirical examples in Wuhan and Italy that seem far worse). The solution is to get better data and then make the appropriate policy response.

    • This addresses the principles of uncertainty. When there is no certainty, then a decision (good or bad in the end, and that can never be ascertained ahead of time) has to be made. Dragging one’s feet does not help either. So while blame is being apportioned to both sides of the argument, in the end, ultracaution is needed. And, very importantly, the perception is risk is what one needs to act upon.

  • “The Greatest Generation” got that name thru living and working hard thru enormous adversity, they didn’t sit around and jack their jaws about whose fault Pearl Harbour was, they knew where it came from

Comments are closed.