In other words, combining positive and negative results from the two tests in the same statistic, as Virginia has done, makes no sense.
But commonwealth officials say they have no choice. Other states are mixing their results, claimed Clark Mercer, the chief of staff to Governor Ralph Northam, at a press conference this week.
“You can’t win” by keeping viral and antibody findings separate in public data, he said, adding that combining the two tests’ results was the only way to improve Virginia’s position in a list of states ranked by the number of tests they had conducted per capita. “If another state is including serological tests, and they’re ranked above Virginia, and we are not, and we’re getting criticized for that, [then], hey, you can’t win either way. Now we are including them, and our ranking will be better, and we’re being criticized,” he said.
We could not find evidence that other states are blending test results in the way that Mercer claimed. In an email, a spokesperson for the Virginia Department of Health claimed that Arizona, West Virginia, and the District of Columbia also mingled viral and antibody results. This is false: Those three governments either separate out, or do not report, the result of negative antibody tests to the public.
Other states report positive serological tests as “probable” COVID-19 cases. This is in line with recommendations published by the Council of State and Territorial Epidemiologists, a nonprofit that works with state and local epidemiologists in the U.S. At least 16 U.S. states and two territories have reported such “probable” cases, although they may not always have done so using serological tests, according to the CDC. However, Arizona, which reports a substantial number of positive serological tests on its own website, does not show up on the CDC’s list. Nor does Kansas, which explicitly states that they are including such probable cases.
While including antibody tests in a state’s total creates too rosy of a testing picture for a state, reporting only probable positive cases without disclosing how many antibody tests are being completed could actually make the situation look more dire in a state than it is.
The spokesperson said that Virginia planned to “disaggregate” its viral and antibody results in the future, but he did not provide a firm date.
Kathy Turner, deputy state epidemiologist for Idaho and the presenting author of the CSTE standards document, did not criticize Virginia’s decision, but she did lay out why her own state decided to keep PCR and serology tests separate.
“[I]n Idaho, we have decided to only display viral tests because those are the denominator we use to calculate our percent positivity rate and we are very confident what they mean,” she told us. “Additionally, we focus on the PCR tests because we can compare the percent positivity over time—before serology tests were available.”
Blending the results also misstates Virginia’s success at improving this crucial metric, sometimes called the “test-positivity rate.” This measurement compares the number of people who have tested positive for the coronavirus to the number of people who have been tested overall. In April, one in five Americans who received PCR tests for the virus were found to be infected, a very high rate that suggested only the sickest people could get a test. For the past week, fewer than one in 10 tests in the U.S. have found a positive result, according to state data. Some of this improvement is certainly the result of the New York metro area’s waning outbreak.
Leaders in many states, including Virginia, have cited the local test-positivity rate to justify loosening shelter-in-place restrictions. Northam has repeatedly said that Virginia’s test-positivity rate had to fall for 14 days before he would loosen restrictions.
But because Virginia combines viral and antibody results, its positivity rate is unusable, said Jha, the Harvard professor. The positivity metric is only useful when describing the result of viral tests, because it is meant to provide a rough estimate of how many people infected with the coronavirus are getting tested for it. Antibody tests, which are meant to sample a broad swath of the healthy population, should not be included in it. By lumping the two tests together, as Virginia has done, states can artificially improve their test-positivity rate.
Only by keeping the two types of test separate can the country—and the commonwealth of Virginia—understand the true scope of its outbreak, experts say.
“You’re comparing apples to pears,” Gigi Gronvall, a senior scholar at the Johns Hopkins Center for Health Security, told us. Viral and antibody tests “look a little bit alike, but it doesn’t let you make the comparison needed. So why not keep them separate?” Gronvall has written about the need to expand antibody testing across the U.S.
She is also worried that the data are statistically meaningless, because viral tests have fewer false-positive errors than antibody tests. “There’s so much variability in the antibody tests that it’s like taking an iffy number and throwing it in with some more reliable numbers,” she said.
Although combining the data from two different tests may seem like a technicality, the decision threatens to confound some of the most important questions about the coronavirus’s path in the United States. How many people are sick right now? How many people can the U.S. actually test for the coronavirus every week? Is the situation outside the New York metropolitan area getting better or worse? Answering these questions requires stable data about how many people have received a diagnostic test and how many of those people are infected.
The scope of the test-mixing problem is not yet clear. No other state aside from Virginia has admitted to counting antibody tests in their overall totals. We do know, however, that large numbers of antibody tests are being completed in the U.S. but not reported to the public by most states. As test numbers have shot up, Quest Diagnostics—one of the two largest commercial laboratories in the country—reported doing only 200,000 PCR tests from May 4 to May 11, which is 180,000 fewer diagnostic tests than in the preceding week. In fact, from May 4 to May 11, the company did almost 100,000 more antibody tests than PCR ones. LabCorp, the other major commercial reference laboratory, has not released similar data, but it has stated that its diagnostic- and antibody-test capacity are about equal.
Two states do report viral and antibody tests separately. In Colorado, 30 percent of tests completed so far in May were for antibodies. In Arizona, 23 percent of the total number of the tests ever done in the state have been serological. In both states, antibody tests started to be conducted en masse around April 26.
This timeline matches up disturbingly well with the improvement of the national picture, which saw a sudden jump at the end of April from an average of about 150,000 tests per day to 200,000, 250,000, and now 300,000 tests per day. The White House has celebrated the improvements in testing, noting in a press conference this week that the United States does more tests per day than any other country. “We have met the moment and we have prevailed,” President Trump said.
In fact, the U.S. has tested a smaller share of its population than other industrialized countries, including Italy, Canada, and Germany.
“I find our testing record nothing to celebrate whatsoever,” Senator Mitt Romney, a Republican of Utah, told Brett Giroir, the assistant secretary for health, at a hearing yesterday. “You celebrated that we had done more tests, and more tests per capita even, than South Korea. But you ignored the fact that they accomplished theirs at the beginning of the outbreak, while we treaded water during February and March.”
It’s possible that Virginia is alone in its reporting methodology, but until we know how many states are dumping antibody tests in their totals, the White House’s claims that the U.S. has overcome its testing plateau cannot be given full weight.
It was one thing to go into the outbreak blind because of the lack of testing, as the U.S. did. It’s another to choose to cloud our vision. Antibody-testing data are an important part of understanding the outbreak; PCR diagnostic-testing data are also an important part of understanding the outbreak. But if states mix the two together, the value of that information plummets.
The good news is that laboratories do report the type of tests they’ve conducted to state governments and the CDC. All states should have that data at their disposal. So should all their residents—and all Americans.
