Months into the coronavirus outbreak, no one really knows how well many of the screening tests work, and experts at top medical centers say it is time to do the studies to find out.
When the new virus began spreading, the Food and Drug Administration used its emergency powers to OK scores of quickly devised tests, based mainly on a small number of lab studies showing they could successfully detect the virus.
That’s very different from the large patient studies that can take weeks or months, which experts say are needed to provide a true sense of testing accuracy.
The FDA’s speedy response came after it was initially criticized for delaying the launch of new tests during a crisis and after the Centers for Disease Control and Prevention stumbled in getting its own test out to states.
But with the U.S. outbreak nearly certain to stretch on for months or even years, some experts want the FDA to demand better evidence of the tests’ accuracy so doctors know how many infections might be missed.
There have been more than 2 million confirmed coronavirus cases in the U.S. and more than 115,000 deaths, according to data compiled by Johns Hopkins University. Cases in nearly half of U.S. states are rising.
In recent weeks, preliminary findings have flagged potential problems with some COVID-19 tests, including one used daily at the White House. Faulty tests could leave many thousands of Americans with the incorrect assumption that they are virus-free, contributing to new flare-ups of the disease as communities reopen.
“In the beginning, the FDA was under a lot of pressure to get these tests onto the marketplace,” said Dr. Steven Woloshin of Dartmouth College. “But now that there are plenty of tests out there, it’s time for them to raise the bar.”
The FDA said in a statement that it has already asked multiple test makers to do follow-up accuracy studies, although it didn’t say for how many of the more than 110 authorized screening tests. The agency also said it is tracking reports of problems. Accuracy has also been an issue with blood tests that look for signs of past infections.
No screening test is 100% accurate. So details on accuracy are routinely provided for tests of all types, including seasonal flu, hepatitis, HIV and cancers. For example, rapid flu tests are known to miss 20% or more of all cases, a factor doctors weigh when treating patients who have symptoms but test negative.
For now, most COVID-19 tests in the U.S. don’t give data on real-world performance, including how often the tests falsely clear patients of infection or falsely detect the virus. That information is lacking for all but a few of the roughly 80 commercial screening tests available, according to an Associated Press review.
The government’s emergency authorization process “requires a lower level of evidence,” the agency said. Makers need only show that a test “may be effective” instead of the usual requirement to demonstrate “safety and effectiveness.” They would have to meet that higher threshold once the U.S. government declares the emergency over.
Many of the commercial test makers submitted results from 60 samples, the minimum number required and mostly used lab-produced specimens of the virus. The FDA now recommends the use of nasal swabs or other real samples from people screened for coronavirus.
Experts say larger patient studies patients are needed to assess a test’s true performance.
Lab testing bears little resemblance to actual — sometimes imperfect — conditions at hospitals, clinics and testing sites noted Dr. Robert Kaplan of Stanford University.
“You’re testing people in parking lots, the patients themselves are extremely anxious and often unable to follow instructions,” said Kaplan, a former associate director of the National Institutes of Health.
Kaplan and others say those differences could explain why some tests are not performing as expected.
Last month, the FDA warned doctors of a potential accuracy problem with Abbott Laboratories’ rapid ID Now test, which delivers results in roughly 15 minutes. The test has been lauded by President Donald Trump and used to screen the president, his staff and visitors to the White House.
The FDA alert followed a preliminary report by New York University that found Abbott’s test missed between a third to one-half of infections caught by a rival test in patients screened for the virus.
Abbott rejected the findings, saying the researchers did not follow the instructions for using its test. The company pointed to alternate patient studies, including its own, that have found the test successfully detects between 91% and 95% or more of virus cases when compared to other tests.
But similar problems with ID NOW’s accuracy have been flagged in preliminary reports by researchers at Stanford University, Loyola University and the Cleveland Clinic.
For now, the FDA is requiring Abbott to conduct follow-up studies in several different patient groups.
The FDA’s emergency standards “are still high but there is a significant difference in the body of work that what would go into a submission under the normal process,” said Abbott vice president John Hackett. “Our normal process takes years to bring these out.”
Requiring bigger studies of all coronavirus tests could provide valuable information, but it could also strain the FDA’s already stretched staff and resources, said Dr. Daniel Schultz, former director of the FDA’s medical device center.
Dr. Colin West of the Mayo Clinic worries doctors and patients have put too much confidence in the current crop of tests, when an unknown number of patients with COVID-19 are likely receiving false negative results.
Even a modest error rate can have grave consequences during an outbreak like COVID-19. West gives the example of a test that is 95% accurate at detecting the virus and is used on 1 million people. That would still result in 50,000 people being incorrectly told that they don’t have the virus.
“The negative test does not mean that I’m off the hook,” West said. “We just need to maintain that level of vigilance until we have a better sense of how good these tests really can be.”
Note to readers: if you purchase something through one of our affiliate links we may earn a commission.
