The current coronavirus disease, Covid-19, has been called a once-in-a-century pandemic. But it may also be a once-in-a-century evidence fiasco.

At a time when everyone needs better information, from disease modelers and governments to people quarantined or just social distancing, we lack reliable evidence on how many people have been infected with SARS-CoV-2 or who continue to become infected. Better information is needed to guide decisions and actions of monumental significance and to monitor their impact.

Draconian countermeasures have been adopted in many countries. If the pandemic dissipates — either on its own or because of these measures — short-term extreme social distancing and lockdowns may be bearable. How long, though, should measures like these be continued if the pandemic churns across the globe unabated? How can policymakers tell if they are doing more good than harm?

advertisement

Vaccines or affordable treatments take many months (or even years) to develop and test properly. Given such timelines, the consequences of long-term lockdowns are entirely unknown.

The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable. Given the limited testing to date, some deaths and probably the vast majority of infections due to SARS-CoV-2 are being missed. We don’t know if we are failing to capture infections by a factor of three or 300. Three months after the outbreak emerged, most countries, including the U.S., lack the ability to test a large number of people and no countries have reliable data on the prevalence of the virus in a representative random sample of the general population.

advertisement

This evidence fiasco creates tremendous uncertainty about the risk of dying from Covid-19. Reported case fatality rates, like the official 3.4% rate from the World Health Organization, cause horror — and are meaningless. Patients who have been tested for SARS-CoV-2 are disproportionately those with severe symptoms and bad outcomes. As most health systems have limited testing capacity, selection bias may even worsen in the near future.

The one situation where an entire, closed population was tested was the Diamond Princess cruise ship and its quarantine passengers. The case fatality rate there was 1.0%, but this was a largely elderly population, in which the death rate from Covid-19 is much higher.

Projecting the Diamond Princess mortality rate onto the age structure of the U.S. population, the death rate among people infected with Covid-19 would be 0.125%. But since this estimate is based on extremely thin data — there were just seven deaths among the 700 infected passengers and crew — the real death rate could stretch from five times lower (0.025%) to five times higher (0.625%). It is also possible that some of the passengers who were infected might die later, and that tourists may have different frequencies of chronic diseases — a risk factor for worse outcomes with SARS-CoV-2 infection — than the general population. Adding these extra sources of uncertainty, reasonable estimates for the case fatality ratio in the general U.S. population vary from 0.05% to 1%.

STAT Reports: STAT’s guide to interpreting clinical trial results

That huge range markedly affects how severe the pandemic is and what should be done. A population-wide case fatality rate of 0.05% is lower than seasonal influenza. If that is the true rate, locking down the world with potentially tremendous social and financial consequences may be totally irrational. It’s like an elephant being attacked by a house cat. Frustrated and trying to avoid the cat, the elephant accidentally jumps off a cliff and dies.

Could the Covid-19 case fatality rate be that low? No, some say, pointing to the high rate in elderly people. However, even some so-called mild or common-cold-type coronaviruses that have been known for decades can have case fatality rates as high as 8% when they infect elderly people in nursing homes. In fact, such “mild” coronaviruses infect tens of millions of people every year, and account for 3% to 11% of those hospitalized in the U.S. with lower respiratory infections each winter.

These “mild” coronaviruses may be implicated in several thousands of deaths every year worldwide, though the vast majority of them are not documented with precise testing. Instead, they are lost as noise among 60 million deaths from various causes every year.

Although successful surveillance systems have long existed for influenza, the disease is confirmed by a laboratory in a tiny minority of cases. In the U.S., for example, so far this season 1,073,976 specimens have been tested and 222,552 (20.7%) have tested positive for influenza. In the same period, the estimated number of influenza-like illnesses is between 36,000,000 and 51,000,000, with an estimated 22,000 to 55,000 flu deaths.

Note the uncertainty about influenza-like illness deaths: a 2.5-fold range, corresponding to tens of thousands of deaths. Every year, some of these deaths are due to influenza and some to other viruses, like common-cold coronaviruses.

In an autopsy series that tested for respiratory viruses in specimens from 57 elderly persons who died during the 2016 to 2017 influenza season, influenza viruses were detected in 18% of the specimens, while any kind of respiratory virus was found in 47%. In some people who die from viral respiratory pathogens, more than one virus is found upon autopsy and bacteria are often superimposed. A positive test for coronavirus does not mean necessarily that this virus is always primarily responsible for a patient’s demise.

If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths. This sounds like a huge number, but it is buried within the noise of the estimate of deaths from “influenza-like illness.” If we had not known about a new virus out there, and had not checked individuals with PCR tests, the number of total deaths due to “influenza-like illness” would not seem unusual this year. At most, we might have casually noted that flu this season seems to be a bit worse than average. The media coverage would have been less than for an NBA game between the two most indifferent teams.

Some worry that the 68 deaths from Covid-19 in the U.S. as of March 16 will increase exponentially to 680, 6,800, 68,000, 680,000 … along with similar catastrophic patterns around the globe. Is that a realistic scenario, or bad science fiction? How can we tell at what point such a curve might stop?

The most valuable piece of information for answering those questions would be to know the current prevalence of the infection in a random sample of a population and to repeat this exercise at regular time intervals to estimate the incidence of new infections. Sadly, that’s information we don’t have.

In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work. School closures, for example, may reduce transmission rates. But they may also backfire if children socialize anyhow, if school closure leads children to spend more time with susceptible elderly family members, if children at home disrupt their parents ability to work, and more. School closures may also diminish the chances of developing herd immunity in an age group that is spared serious disease.

This has been the perspective behind the different stance of the United Kingdom keeping schools open, at least until as I write this. In the absence of data on the real course of the epidemic, we don’t know whether this perspective was brilliant or catastrophic.

Flattening the curve to avoid overwhelming the health system is conceptually sound — in theory. A visual that has become viral in media and social media shows how flattening the curve reduces the volume of the epidemic that is above the threshold of what the health system can handle at any moment.

Yet if the health system does become overwhelmed, the majority of the extra deaths may not be due to coronavirus but to other common diseases and conditions such as heart attacks, strokes, trauma, bleeding, and the like that are not adequately treated. If the level of the epidemic does overwhelm the health system and extreme measures have only modest effectiveness, then flattening the curve may make things worse: Instead of being overwhelmed during a short, acute phase, the health system will remain overwhelmed for a more protracted period. That’s another reason we need data about the exact level of the epidemic activity.

One of the bottom lines is that we don’t know how long social distancing measures and lockdowns can be maintained without major consequences to the economy, society, and mental health. Unpredictable evolutions may ensue, including financial crisis, unrest, civil strife, war, and a meltdown of the social fabric. At a minimum, we need unbiased prevalence and incidence data for the evolving infectious load to guide decision-making.

In the most pessimistic scenario, which I do not espouse, if the new coronavirus infects 60% of the global population and 1% of the infected people die, that will translate into more than 40 million deaths globally, matching the 1918 influenza pandemic.

The vast majority of this hecatomb would be people with limited life expectancies. That’s in contrast to 1918, when many young people died.

One can only hope that, much like in 1918, life will continue. Conversely, with lockdowns of months, if not years, life largely stops, short-term and long-term consequences are entirely unknown, and billions, not just millions, of lives may be eventually at stake.

If we decide to jump off the cliff, we need some data to inform us about the rationale of such an action and the chances of landing somewhere safe.

John P.A. Ioannidis is professor of medicine and professor of epidemiology and population health, as well as professor by courtesy of biomedical data science at Stanford University School of Medicine, professor by courtesy of statistics at Stanford University School of Humanities and Sciences, and co-director of the Meta-Research Innovation Center at Stanford (METRICS) at Stanford University.

  • I totally agree with your comment and I add : the experience in italy and madrid is worthless ? social restrictions were not accomplish and exponential curve cases rapidly ended up with hospital capacities, even if realistic data were possible is worthless in the middle of an exponential curve. we are talking of EPIDEMIC , not ENDEMIC for christ sake!
    and with a flattened curve you:
    1) buy some time to develop a vaccine
    2) treat properly covid patients
    3) treat properly not covid patients
    of course there will be giant economic losses, perhaps an eldery people mass killing machine is what our western over-aged health and social retirement system needs, and I write from Montevideo, Uruguay

  • This article is brilliant, overdue and cause for hope if heeded.
    Thank you
    I am sending it on to all on my contact lists

  • This is 2020 and not 2018. What we have now and what we didn’t have then is: Central Heat, Potable water, water sanitation, Refrigeration, Canning, Freezing and preserving of foods. Homes have washers and dryers and soap is readily available and used and buildings often have ventilation systems to exchange air in the building every so many minutes. Add this technology to foods that have iodine, B vitamins and trace minerals added to foods and individuals taking sound nutrition, avoiding refined white flour and sugars and making an effort to improve the health of the host and you have a modern population that is potentially much hardier than the population of 1918.
    Social distancing is good, washing hands is awesome, drinking plenty of fresh pure water is great, walking and sunlight is incredible, bathing, washing and laundering clothes desired and needed, eating spinach, broccoli, cauliflower, cabbage, kale, and adding onion, and garlic to stews and soups and taking homeopathics tinctures, and digesting foods and getting pro-tease enzymes will greatly help the host to raise their immune system. TW

  • Too many conflicting sciences and/or opinions floating around, nobody knows everything.  

    We’ve never experienced anything like this, at the rate it is moving, due to the mobility of our societies and if you can limit your exposure to an unknown contaminated object or person, why wouldn’t you chose to do so?

    Does anyone ever make a wager on a sporting event without knowing the odds or the records of the two opposing teams?  No, not too often.  And in this case I wouldn’t.  Because the loser doesn’t just lose their money, they may unfortunately lose their life or sicken others.

  • Is it reasonable to project Diamond Princess numbers directly to the general population? I’d expect the average cruise passenger to be healthier than the average person of their age. (just by virtue of being able to take a cruise)

  • You’re making assumptions based on extremely limited information. That’s the danger that the article correctly highlighted.

    • No assumptions were made. Using an available data set and correcting for variances that would occur when expanding it upon a larger population he came up with a range of more likely death rates. Very simple, yet useful, statistics. If we went off of the “confirmed” influenza cases this season to assess the death rate of the common flu we would have a number that is close to 10%. But obviously way more people contracted influenza than just those who tested positive, which is where stats comes in to provide a more accurate number and thus more accurate death rate. Yet we are currently using that same flawed measurement for the covid19 death rate (Confirmed cases only). The point made about mild Coronavirus cases being untraceable is interesting. The rate that these Covid19 tests are being developed leaves the possibility of false positives bc there is no way to control for every possible strain of Coronavirus to ensure specificity of the primers. Right now the media panic is based on a death rate that is profoundly unreliable, especially when considering that a huge chunk of people who contract the virus experience little to no symptoms. Yes, it is wise to be safe by avoiding close personal contact with strangers and be ever mindful of your sanitation practices. But these are also guidelines that one should always be following, no matter the year or season.

  • Excellent article. If we’re not careful, this could backfire and coronavirus will be the least of our problems.

  • The media continually presents bad news. Why don’t we hear “China is returning to normal, we in the rest of the World also will return to normal”.

    The statistics presented in the media are inconsistent and confusing; and’ as John says, the statistics re Covid-19 are fundamentally unreliable – “Lies, damn lies and statistics”. It is extremely difficult not to question “Have our leaders got it right?”

    The almost totally pessimistic news in the media is causing unnecessary hysteria (hoarding toilet roles!!) and the extreme “flattening the curve” measures causing chaos to world-wide economies.

    • Dusty, I concur. One needs to dig to find that China and SK/Japan to some extent, are already “re-opening”. If I didn’t have colleagues with spouses in China (first hand information) that could then key me into other news sources, our media would have us believe there is no world left in Asia.

      Instead, the Chinese cities on lockdown are experiencing a gradual re-opening, people are now being allowed in public whenever they want (vs. before only being allowed out on certain intervals), restaurants are reopening, airports are in preparation for reopening (not as simple as turning the lights on), even western companies are back to 100% open (such as Starbucks, that was at 50%, and Apple stores, which were completely shut down). Museums and such re-opening in Japan and SK, etc.

      We are barraged with constant headlines that are mostly clickbait and at worst, downright misinformation. The media is still parroting the “80% are mild cases” statistic, which was based on the first 40k patients or so in China (and Hubei at that which we know had a much higher impact than even the rest of China) – yet we now know that worldwide it’s 94%. That’s one hell of a difference from 80%! Or, how about the media report the CFR by age group rather than a bulk statistic? While I feel for those 80+ that have a roughly 1 in 7 chance of dying from this, for the rest of the population under 60, the confirmed CFR is perhaps a few times that of the flu.

      Lastly, buried in the back pages of a major new org I found today a story that indicated modeling concludes that 6 of 7 cases are “unreported”. While the study and story don’t conclude as such, that could bring the real CFR of this crashing down [in general I find reputable sources of data very hesitant to report a CFR; heck the CDC doesn’t even report a CFR for the seasonal flu].

  • While I agree that more data is obviously needed, the data analysis in this article is abysmal and irresponsible coming from a Stanford professor.

    Why is your worst-case, so-ridiculous-it’s-not-endorsed scenario a 1% death rate when Italy stands at 8% today? Do you have evidence that the testing bias produces an 8x overestimation? Or, is your estimate based on an extrapolation from 700 mostly Japanese and affluent patients who do not have the same rates of diabetes, heart disease or obesity as the American public?

    If our hospitals are overwhelmed, the data so far shows that many more than 1% of diagnosed patients will die. Flattening the curve, from my understanding, will not necessarily change the overall number of cases. Spreading those cases out so that hospitals are overwhelmed for longer will by definition save lives as our resources are put to maximal use for more time. Yes, a higher proportion of deaths would be from non-covid patients who couldn’t access care, but less people would die overall. I don’t see how this would be ‘making things worse’.

    It’s important to consider all possibilities, but this does not read like a good-faith effort to constructively criticize America’s response to this situation. Given the (until very recently) dismissive attitude of our government towards this crisis and the lingering doubts/conspiracy theories abounding in our population, we can’t afford to have poorly-constructed devil’s advocate-style articles trying to grab people’s attention right now.

    I hope that this article simply produces some intellectual discussion and further careful consideration of policies by its readers. Looking at some of the responses here and on Twitter though, I fear that it will ironically be used as ammunition to reinforce dangerous and ignorant attitudes in a time of poor information and high risk. Your readers deserve better.

    • When you say Italy has an 8% death rate, I think you’re dividing the number of deaths by the number of known cases. But isn’t it likely that there have been many more cases that have not been identified? If so, 8% would be an overestimate for the actual death rate. When Professor Ioannidis refers to a 1% death rate, he’s guessing about the percent of all cases, known and unknown, that result in death.

    • I guess you anticipated this objection when you wrote, “Do you have evidence that the testing bias produces an 8x overestimation?” I don’t know of evidence for that conclusion, but I don’t see why we should rule it out.

    • Just looking at the numbers of sick and dead in China tells me that this article is right. China has had the virus since November and they only had slightly over 3,000 dead. Based on the pessimistic numbers from the media their streets should be covered in dead bodies.

    • It’s common knowledge by now that:
      – Italy has a relatively old population – second oldest only to Japan. Consider a nursing home. If a highly infectious disease that has a 50% CFR in those above 80 but 0 below that were to hit a nursing home, you’d expect there to be a high number of deaths relative to the general population. This is what we are seeing with Covid except the CFR seems to be on the order of 15% for those 80+. Comorbidities, which I’d say the vast majority of the elderly have, increase that CFR as well.
      – Italy has a relatively low per-capita ICU bed capacity especially when you consider that something like 80-90% capacity is achieved in any random winter due to the flu and other diseases. So, to claim that Italy’s ICU’s are being overrun because of Covid is a bit disingenuous.
      – In general, we know there is a huge undercurrent of asymptomatic people. This should be obvious as symptomatic cases pop up seemingly at random in people with no known risk factors. This was confirmed in a model with results reporting today, that we’re only seeing about 1 in 7 total cases; that is, up to 85% of cases are unreported/asymptomatic/etc. This of course would crash the CFR if the denominator in the calculation were multiplied by 5-7.

Comments are closed.

A roundup of STAT’s top stories of the day in science and medicine

Privacy Policy