Skip to Main Content

The current coronavirus disease, Covid-19, has been called a once-in-a-century pandemic. But it may also be a once-in-a-century evidence fiasco.

At a time when everyone needs better information, from disease modelers and governments to people quarantined or just social distancing, we lack reliable evidence on how many people have been infected with SARS-CoV-2 or who continue to become infected. Better information is needed to guide decisions and actions of monumental significance and to monitor their impact.

Draconian countermeasures have been adopted in many countries. If the pandemic dissipates — either on its own or because of these measures — short-term extreme social distancing and lockdowns may be bearable. How long, though, should measures like these be continued if the pandemic churns across the globe unabated? How can policymakers tell if they are doing more good than harm?

advertisement

Vaccines or affordable treatments take many months (or even years) to develop and test properly. Given such timelines, the consequences of long-term lockdowns are entirely unknown.

The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable. Given the limited testing to date, some deaths and probably the vast majority of infections due to SARS-CoV-2 are being missed. We don’t know if we are failing to capture infections by a factor of three or 300. Three months after the outbreak emerged, most countries, including the U.S., lack the ability to test a large number of people and no countries have reliable data on the prevalence of the virus in a representative random sample of the general population.

advertisement

This evidence fiasco creates tremendous uncertainty about the risk of dying from Covid-19. Reported case fatality rates, like the official 3.4% rate from the World Health Organization, cause horror — and are meaningless. Patients who have been tested for SARS-CoV-2 are disproportionately those with severe symptoms and bad outcomes. As most health systems have limited testing capacity, selection bias may even worsen in the near future.

The one situation where an entire, closed population was tested was the Diamond Princess cruise ship and its quarantine passengers. The case fatality rate there was 1.0%, but this was a largely elderly population, in which the death rate from Covid-19 is much higher.

Projecting the Diamond Princess mortality rate onto the age structure of the U.S. population, the death rate among people infected with Covid-19 would be 0.125%. But since this estimate is based on extremely thin data — there were just seven deaths among the 700 infected passengers and crew — the real death rate could stretch from five times lower (0.025%) to five times higher (0.625%). It is also possible that some of the passengers who were infected might die later, and that tourists may have different frequencies of chronic diseases — a risk factor for worse outcomes with SARS-CoV-2 infection — than the general population. Adding these extra sources of uncertainty, reasonable estimates for the case fatality ratio in the general U.S. population vary from 0.05% to 1%.

STAT Reports: STAT’s guide to interpreting clinical trial results

That huge range markedly affects how severe the pandemic is and what should be done. A population-wide case fatality rate of 0.05% is lower than seasonal influenza. If that is the true rate, locking down the world with potentially tremendous social and financial consequences may be totally irrational. It’s like an elephant being attacked by a house cat. Frustrated and trying to avoid the cat, the elephant accidentally jumps off a cliff and dies.

Could the Covid-19 case fatality rate be that low? No, some say, pointing to the high rate in elderly people. However, even some so-called mild or common-cold-type coronaviruses that have been known for decades can have case fatality rates as high as 8% when they infect elderly people in nursing homes. In fact, such “mild” coronaviruses infect tens of millions of people every year, and account for 3% to 11% of those hospitalized in the U.S. with lower respiratory infections each winter.

These “mild” coronaviruses may be implicated in several thousands of deaths every year worldwide, though the vast majority of them are not documented with precise testing. Instead, they are lost as noise among 60 million deaths from various causes every year.

Although successful surveillance systems have long existed for influenza, the disease is confirmed by a laboratory in a tiny minority of cases. In the U.S., for example, so far this season 1,073,976 specimens have been tested and 222,552 (20.7%) have tested positive for influenza. In the same period, the estimated number of influenza-like illnesses is between 36,000,000 and 51,000,000, with an estimated 22,000 to 55,000 flu deaths.

Note the uncertainty about influenza-like illness deaths: a 2.5-fold range, corresponding to tens of thousands of deaths. Every year, some of these deaths are due to influenza and some to other viruses, like common-cold coronaviruses.

In an autopsy series that tested for respiratory viruses in specimens from 57 elderly persons who died during the 2016 to 2017 influenza season, influenza viruses were detected in 18% of the specimens, while any kind of respiratory virus was found in 47%. In some people who die from viral respiratory pathogens, more than one virus is found upon autopsy and bacteria are often superimposed. A positive test for coronavirus does not mean necessarily that this virus is always primarily responsible for a patient’s demise.

If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths. This sounds like a huge number, but it is buried within the noise of the estimate of deaths from “influenza-like illness.” If we had not known about a new virus out there, and had not checked individuals with PCR tests, the number of total deaths due to “influenza-like illness” would not seem unusual this year. At most, we might have casually noted that flu this season seems to be a bit worse than average. The media coverage would have been less than for an NBA game between the two most indifferent teams.

Some worry that the 68 deaths from Covid-19 in the U.S. as of March 16 will increase exponentially to 680, 6,800, 68,000, 680,000 … along with similar catastrophic patterns around the globe. Is that a realistic scenario, or bad science fiction? How can we tell at what point such a curve might stop?

The most valuable piece of information for answering those questions would be to know the current prevalence of the infection in a random sample of a population and to repeat this exercise at regular time intervals to estimate the incidence of new infections. Sadly, that’s information we don’t have.

In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work. School closures, for example, may reduce transmission rates. But they may also backfire if children socialize anyhow, if school closure leads children to spend more time with susceptible elderly family members, if children at home disrupt their parents ability to work, and more. School closures may also diminish the chances of developing herd immunity in an age group that is spared serious disease.

This has been the perspective behind the different stance of the United Kingdom keeping schools open, at least until as I write this. In the absence of data on the real course of the epidemic, we don’t know whether this perspective was brilliant or catastrophic.

Flattening the curve to avoid overwhelming the health system is conceptually sound — in theory. A visual that has become viral in media and social media shows how flattening the curve reduces the volume of the epidemic that is above the threshold of what the health system can handle at any moment.

Yet if the health system does become overwhelmed, the majority of the extra deaths may not be due to coronavirus but to other common diseases and conditions such as heart attacks, strokes, trauma, bleeding, and the like that are not adequately treated. If the level of the epidemic does overwhelm the health system and extreme measures have only modest effectiveness, then flattening the curve may make things worse: Instead of being overwhelmed during a short, acute phase, the health system will remain overwhelmed for a more protracted period. That’s another reason we need data about the exact level of the epidemic activity.

One of the bottom lines is that we don’t know how long social distancing measures and lockdowns can be maintained without major consequences to the economy, society, and mental health. Unpredictable evolutions may ensue, including financial crisis, unrest, civil strife, war, and a meltdown of the social fabric. At a minimum, we need unbiased prevalence and incidence data for the evolving infectious load to guide decision-making.

In the most pessimistic scenario, which I do not espouse, if the new coronavirus infects 60% of the global population and 1% of the infected people die, that will translate into more than 40 million deaths globally, matching the 1918 influenza pandemic.

The vast majority of this hecatomb would be people with limited life expectancies. That’s in contrast to 1918, when many young people died.

One can only hope that, much like in 1918, life will continue. Conversely, with lockdowns of months, if not years, life largely stops, short-term and long-term consequences are entirely unknown, and billions, not just millions, of lives may be eventually at stake.

If we decide to jump off the cliff, we need some data to inform us about the rationale of such an action and the chances of landing somewhere safe.

John P.A. Ioannidis is professor of medicine and professor of epidemiology and population health, as well as professor by courtesy of biomedical data science at Stanford University School of Medicine, professor by courtesy of statistics at Stanford University School of Humanities and Sciences, and co-director of the Meta-Research Innovation Center at Stanford (METRICS) at Stanford University.

  • I have been asking a similar question since the arrival of Covid19 earlier this month. The CDC and WHO keep adding numbers of cases as if, it’s a surprise that they are going up despite efforts to control, with shut downs etc.
    We are testing more and more the increase shouldn’t surprise anyone
    My contention all along is that if 80% of cases are mild. Then it only stands to reason that most people who get covid19, are not tested. Meaning that as of this time the number of recognized case is around 200,000. It is more realistic that it is far greater than that, 2-3 or more times. This article is one of the first that I have seen where the author suggests that we don’t know how many people have, or have had covid19
    I wish that this were addressed by more people who have microphones in their faces.
    We should use best practices for controlling the spread but I’m not so sure we are.

  • The author suggests this may be a non material flu. I suggest he reads the data about deaths in Italy and see that, left to itself, Covid 19 is causing thousands of extra deaths that are never seen in seasonal flu. In Bergamo, a highly hit area, obituaries are 10 times what they were last year.

    • Really 10X more in Italy? Where did you get that #? my research say 3million people a year get the flu and 230,000-280,000 die. 10x would mean of half their total population has gotten this virus and 2.5million have died. Math is not adding up.

    • Jimmy – please provide a link supporting that 230k to 280k people in Italy die annually from the flu. I can’t readily find that number with Google, and it sounds wildly high.

      The U.S. has about 5.5x the population of Italy and U.S. annual flu deaths are estimated to have been between 12k and 61k for the past decade – https://www.cdc.gov/flu/about/burden/index.html

    • Italy is higher because age demographic (one of the older in the world) and culture. Italians like to get close. Now add underlining conditions which in the past would have killed them but is treated with pills. Then add a bad health system and it spells disaster. Thats not taking away how bad an infection spread this is. But 2 years ago when the Flu killed ~80000 Americans with 90% of that being older then 60 we didn’t see this hysteria.

  • I found this very reassuring until I went back and read it again. As the previous commenter points out, he’s presuming only 1% of the population gets infected… then coming up with a low number of actual deaths based on a measured estimate of mortality rate. But where is the evidence to suggest that only 1% will be infected? This is far, far lower than any other projection I’ve seen…

  • This is the best article I’ve read to-date and it sums up nicely my position on where we should go from here. You have put eloquent words to my similar thoughts.

  • I am not a statistician but I have a question.

    This is a very logical conclusion with an important presumption : health care systems have no limits. Since that does not seem to be the case in reality, isn’t it biased even a little bit?

    • He doesn’t assume that. It’s one of the factors he explicitly deals with when he discusses why “flattening the curve” could result in excess deaths as it prolongs the time the health care system cannot keep up with demand and that COVID-19 prevents it from taking care of other causes of mortality.

      Since he explicitly accounts for that, I’m not clear why you assume he presumes that health care systems have no limits.

    • Avoiding the lockdown might send thousands of additional patients to the hospitals in a very short period of time, potentially overheating them like in Italy or in the Eastern part of France.

      It’s true that it will result in a longer crisis. But we are still not sure if and for how long an immunity is developed after someone is infected.

      Besides, it seems to have worked in China where the number of new cases decreased drastically afterwards. Maybe I’m wrong.

  • The author is just talking about deaths…up to 20% of the cases are severe, potentially requiring intensive care treatment and ventilation. As long as there are enough ICU beds, most death can be prevented. But when ICU capacity is gone deaths will increase dramatically.

  • Great great great description
    It addresses two of my concerns
    1) are people dying with it or from it? The Info is far from convincing about the latter
    2) are the tests being used exclusive enough for the strain of Covid 19? There are are a lot of corona virus carriers, but of the less threatening variety.
    JCGroh MD

    • That is a very good question when 80 percent of the RNA in Covid 19 is the same as for the previous SARS event. The PCR primers need to be specific to Covid 19.

      The second question is how many people have had covid 19 or a close relative of it, recovered and never got tested. This can be determined easily via ELISA testing which has not been happening in the USA so far.

  • As a longtime virologist, I’m pleased to see this impatient appeal for better data with which to make upcoming decisions. Many readers will misinterpret: trying to “flatten the curve” was not a bad decision (given existing data), but “flying blind” on upcoming decisions is frustrating. Who to blame? Start with myopic austerity-budgeteers, who drove a trickle-down unpreparedness (CDC and elsewhere) that has colossally backfired. There have been other blunders, along with an inexplicable delay in high-throughput serology assays (to track recent and past infections); anybody who says such assays are hard to develop (in 2020) is blowing smoke. And what geniuses failed to anticipate and avoid the “perfect storm” mixing bowl at airports as travelers returned from Europe? Politicians? Data, please.

  • An incredibly important piece thank you. What is quite frightening is how we are heading into strategies that are going do untold harm. Many people now or already losing their jobs and of course it’s always going to be the poor that suffer most;The ones on struggle day today to make a living and keep going. Self isolation may work for the rich but not the poor. The slums and ghettos of the world wouldn’t be able to do it even if they wanted to. Isolating the elderly is spreading great fear and we know that loneliness is one of the biggest causes of depression suicide not to mention increasing vulnerabilities to physical health problems. What about the recently bereaved what about people who don’t have friends and relations nearby? What about individuals and socially anxious anyway? The long-term Social and psychological damage of the measures are unknown. And as you say on the basis of this uncertain data we are allowing governments to become Draconian. I understand that in Italy police are on the streets and can question you where you are going. We can project to 2035 when we have another more virulent virus and now we need to wear gas masks before we are allowed out, touching and kissing are prohibited and all communication must be done by television screen — we are not thinking clearly about what kind of world we are creating –While promoting paranoia fear and social distancing may have short-term benefits the long-term costs are not even been thought about. We may live in a world where we can to some degree protect ourselves from viruses but it will be unrecognisably human. Any cost that the virus causes has to be set against the enormity of the social and psychological costs, long-term too. This isn’t to argue what should or should not be happening to flatten the curve it’s just to say that we haven’t really discussed the social and psychological consequences of what were doing and what we are accepting.

  • It is fair to wonder what “social distancing” and the near-total shutdown of various industries will do to people. If we increase the suicide rate, due to lack of ability to simply exercise our sociability, or due to loss of jobs, sense of purpose, income, etc., what was our gain? We’re living as if… dead… to avoid… death? No, to avoid getting sick for a week or two. Sure, tell the elderly to stay in and to avoid close contact. Same for people with underlying health conditions that put them at risk. The rest of us? Open the restaurants, bars, Spring Break, MLB, the NBA, and let’s get on with life. We’re Americans; we’re supposed to be free. This is not instant death if you catch it. I’m tired of the hyperbole and folly.

Comments are closed.