A key component to managing the Covid-19 pandemic is frequent, rapid, and routine testing of a large number of Americans — including those without symptoms. With public discussion of this issue has come increased scrutiny of the accuracy of Covid-19 tests, especially the rapid point-of-care tests whose results can come back in a matter of minutes.

Yet these discussions rarely explain what test accuracy actually means, and how it should be used in clinical decision-making. Without this understanding, it is impossible to make truly informed decisions about whether and how these rapid point-of-care tests should be used. And the focus on accuracy obscures an important point: even tests that aren’t perfect can play important roles in controlling this pandemic.

Fortunately, there is a precise way to evaluate the accuracy and interpretation of tests like these. Called Bayes’ Theorem, it can help make sense of the current situation by showing us that the usefulness of a test depends not just on how accurate it is but also on how likely someone is to have the condition it is testing for.

advertisement

Test characteristics are usually reported in terms of their **sensitivity** and **specificity**. Sensitivity, sometimes called the true positive rate, tells you what percentage of people who have the disease you are testing for (like Covid-19) will test positive. Specificity, sometimes called the true negative rate, tells you what percentage of people who don’t have the disease will test negative.

In other words, in a theoretical group of 100 people infected with SARS-CoV-2, the virus that causes Covid-19, how many would have a positive test? And in a theoretical group of 100 people not infected with the virus, how many of would have a negative Covid-19 test?

advertisement

That is all well and good, but in the real world we don’t have 100 theoretical people known to be with or without Covid-19. That’s what we’re trying to find out. Instead, what we have are 100 people with positive or negative test results, and we need to know whether or not they really have the disease based on the testing that they’ve had.

The answers to these questions are known as the **positive predictive value (PPV)** and the **negative predictive value (NPV)**. The positive predictive value represents this: Of 100 people with positive tests, how many of them actually have the disease? The negative predictive value is analogous: Of 100 people with negative tests, how many of them do not have the disease.

Here’s how they are different from sensitivity and specificity: Sensitivity and specificity start with people with and without the disease — which we don’t know — while the positive predictive value and negative predictive value start with people with positive or negative test results — which we do know.

The complicating factor is that positive predictive values and negative predictive values are not just dependent on the characteristics of the test (the sensitivity and specificity) but also on how likely someone is to have the disease in the first place, known as the **pretest probability**. This is the crucial message of Bayes’ theorem, and it’s key to understanding how to interpret the usefulness of a test and how — and how often — the test should be used.

We put all these concepts together with two examples: the first with a low pre-test probability, the second with high pre-test probability. For both examples, let’s set the sensitivity of a rapid point-of-care Covid-19 test at 80%, with a specificity of 100%, reasonable estimates for these sorts of tests. Some criticize the rapid tests for low sensitivity, with people understandably worried that it will miss too many cases. As we’ll show, that can be a problem, but it isn’t always.

In the first example with low pre-test probability, say 1,000 people who feel fine — they are completely asymptomatic — are tested for Covid-19. Such testing is important because people without symptoms can transmit the disease, especially if they become sick soon after. This is the kind of testing proposed for some students entering school, for some employees returning to in-person work, and for entering the state of Maine.

In Massachusetts, where we live and work and where the disease is currently under good control, the pre-test probability in asymptomatic people is low — certainly no higher than 1%.

Do the math using Bayes’ Theorem (you can see it laid out here), and approximately eight of these 1,000 people will have positive rapid tests. All eight will have Covid-19: the positive predictive value is 100%. Of the 992 with negative tests, 990 of them will not have Covid-19 and two of them will: the negative predictive value is 99.8% (990 divided by 992). In other words, if you have a rapid test for Covid-19 and it is negative, you have 2 in a 1,000 chance of actually having — and potentially spreading — the disease. This is a small enough chance to let you attend school, go to work, or visit Maine.

And even though the test isn’t perfect, it’s far better than what we’re doing now, which is testing hardly anyone without symptoms, in part due to concerns about testing accuracy. A rapid, inexpensive Covid-19 test, even one with lower sensitivity — like 80%, meaning it misses 20% of cases — would be a true advance in our efforts to control the pandemic. And there’s this to consider: Those who are most infectious have the highest amount of virus and are the least likely to have falsely negative rapid tests.

On to the second example, among those with a high pre-test probability: 1,000 patients with signs or symptoms of Covid-19, such as fever or shortness of breath. Let’s say the pre-test probability here is 30%, as it was during the surge of cases in Massachusetts this spring.

Do the math using Bayes’ Theorem and the positive predictive value remains at 100% (240 out of 240 with a positive test), but the negative predictive value has dropped to 92% (700 of the 760 with a negative test). Now it’s time to worry about low sensitivity: 8% of those with negative tests actually have Covid-19. That is too high, and it’s why symptomatic patients at our hospital are tested twice —using the most accurate tests available — 24 hours apart, with additional testing if we’re really suspicious.

Applying Bayes’ theorem to Covid-19 testing has several implications for controlling the pandemic. First, even tests with moderate sensitivity, like the current rapid point-of-care tests, are good enough when the pre-test probability of those being tested is low, such as among asymptomatic people in states with low rates of community spread. These tests will be crucial tools for getting back to work and school safely, and arguably attainable even before we have a safe vaccine.

Second, these tests work best when the rate of community transmission is low, as the low pre-test probability means that few cases will be missed. This is one of several reasons why expert recommendations for phased re-opening recommend first getting the disease under control (with a combination of stay-at-home orders and recommendations for physical distancing and wearing face coverings in public) and only then initiating testing on a massive scale for screening and contact tracing.

Third, individuals with symptoms of Covid-19 should be given the most sensitive tests available. As their pre-test probability is high, they may need to be tested twice, or undergo other testing strategies (such as antibody testing, or PCR testing of sputum samples) to ensure they do not have Covid-19.

The goal at this point should be to pursue testing strategies that are rapid, convenient, accessible, and inexpensive. That way a large proportion of the population can be tested. A small decline in the accuracy of Covid-19 tests is less important at this point than our need for testing on a massive scale.

*Jeffrey L. Schnipper is an internal medicine physician, research director of the Division of General Internal Medicine and Primary Care at Brigham and Women’s Hospital in Boston, and professor of medicine at Harvard Medical School. Paul E. Sax is an infectious disease physician, clinical director of the Division of Infectious Diseases at Brigham and Women’s Hospital, and professor of medicine at Harvard Medical School.*

I have yet to see a rapid test with 100% specificity. Even at the Roche lab test’s 99%, it’s false pos galore given 1/1000 prevalence

CORRECTION TO MY 5:18 POST

We know two ways:

1. We know from past testing that about 50% of covid-19 cases are asymptomatic. So if the incidence of symptomatic covid-19 infection is less than 1%, which it is, the rate of asymptomatic infection is about 0.5%. And we know that no community has a current symptomatic infection rate of 1%. Not even the worst cities and states in the world (NY, NJ, CT) had at any point in time 1% of their population currently infected.

2. If the rate of asymptomatic infection was any higher than 1% there would currently be 3.2 million Americans with asymptomatic covid-19. Next week they’d mostly be recovered and another 3.2 million would be asymptomatically infected. In 60 days 60% of the USA’s population would have recovered from covid-19 and be immune for at least a year. We’d have had herd immunity by the end of April and now 4 months later we’d be slowly forgetting covid-19 ever existed.

These ($3 x 7 days/week =) $21 a week tests are infinitely more useful and effective than $300 tests administered once a week because both tests only show you’re not infective that day.

And for the $300 test, by the time you have results that safe day was yesterday.

“the pre-test probability in asymptomatic people is low — certainly no higher than 1%.”

How do you know that?

We know two ways:

1. When the rate of symptomatic infection is about 50% and the incidence of known covid-19 infection is less than 1%, the rate of asymptomatic infection is about 0.5%. In fact the rate of symptomatic infection in the general public is less than 1%.

2. If the rate of asymptomatic infection was any higher than 1% there would currently be 3.2 million Americans with asymptomatic covid-19. Next week they’d mostly be recovered and another 3.2 million would be asymptomatically infected. In 60 days 60% of the USA’s population would have recovered from covid-19 and be immune for at least a year. We’d have had herd immunity by the end of April and now 4 months later we’d be slowly forgetting covid-19 ever existed.

Nonsense. We don’t really know the pre-test probability, and it’s likely much higher than 1%, which renders the rest of the math meaningless (GIGO).

The claim that anything is better than nothing is a false proposition because it assumes we can’t do it properly, despite other countries proving that it can be done properly.

The high pre-test probability case shows why the entire argument is specious. Many of the public won’t do retesting (because they test supposedly showed them to be virus-free), and false negatives may encourage risky behavior.

It shows why doctors should stick to doctoring, and leave science to properly trained scientists.

John,

Please do not write off this solution so quickly. Please see:

New Approaches to Covid-19: Rapid Testing, Herd Immunity, and the Role of Narrative

https://www.youtube.com/watch?v=Ew2MEF4XX8w&t=4099s From 5:58 to

An Update on Covid-19 Testing, Treatments, and Vaccines Jul 30, 2020

https://www.youtube.com/watch?v=kNADw5io9Ms from 5:30 to 27:15

https://www.microbe.tv/twiv/

TWiV 640: Test often, fast turnaround, with Michael Mina July 16, 2020

Journal Preprint

https://www.medrxiv.org/content/10.1101/2020.06.22.20136309v2

If such test are extremely cheap then students might need to pass the test before entering the school, or gym, or whatever.

By the way, being a doctor and being a scientist are not mutually exclusive.

Thank you for your consideration.

> We don’t really know the pre-test probability, and it’s likely much higher than 1%, which renders the rest of the math meaningless (GIGO).

No, it really doesn’t. Plug in your own numbers and see for yourself: https://www.robertjacobson.dev/BayesTesting/BayesTesting.html

What a laugh!? False positive, false negative… But it all helps… Too keep the sheeple scared, the $ coming in & the lie that “science” can solve all of our problems!

We have already had 180,000 deaths in the United States, and you think investing in mitigation is a conspiracy theory.

What is really comical is the notion that these powerful, rich academic fat cats are out there calling the shots. Have you ever met an academic?

1. We’re very much not wealthy.

2. In fact, we spend a ridiculous amount of time clamoring for the few limited scraps available from strained grant funders.

3. We are as close to a modern Cassandra as I can imagine outside of Greek myth. The vast majority of our time interacting with the public is spent trying to get people to believe in facts and being largely ignored.

Of course, it doesn’t matter what I say to someone that deep into conspiracy theories. I am perpetually astonished by the grotesque irony that those who are most easily manipulated by the media are exactly those who live in the most fear of and imagined rebellion against media manipulation.