Testing the tests —
And many of the tests available to the public produced variable results.
John Timmer
–
How much of an immune response does a SARS-CoV-2 infection produce? It’s a critical question for all sorts of reasons. To begin with, long-lasting immunity, either through an infection or a vaccine, is critical for any hope of returning the world to something that resembles its pre-pandemic state. It’s also essential to understanding how safe people who have recovered from infections are and how they can behave in the face of continued outbreaks and spread.
But there are also more subtle public policy issues. Since testing wasn’t generally available at the time of many outbreaks, we’ll need antibody tests to figure out who was actually exposed. And the accuracy of those tests—which has been called into question—can have a big influence on studies of the pandemic’s progression.
A bunch of recent draft papers have looked at the sort of immune response we’re seeing in patients who have cleared the virus after testing positive for it. And the results suggest that it’s very variable—as is the quality of the tests that detect it. (We’ll remind you that pre-publication documents carry some quality risks.)
Testing 1, 2, 3, 4
One of the more comprehensive studies on the subject comes from the New York Blood Center. That organization was serving as the clearinghouse for tests of whether the antibodies of those who have cleared the infection could be used to help patients with COVID-19. So, it had access to a large collection of samples from people who were known to have been infected. A research team based there used those samples to test both the immune response in these individuals and the ability of tests to assay it.
Several of these tests looked at whether antibodies against the proteins of SARS-CoV-2 were present at all. One of those test is called an enzyme-linked immunosorbent assay, or ELISA. This produces a high-quality, quantitative measure of the number of antibodies that recognize a specific protein. But it’s not very amenable to automation, and it tends to require researchers to perform the tests by hand.
There are high-throughput diagnostic tests that work on similar principles to the ELISA, and they are typically used by centralized testing facilities. The researchers had access to these tests from two different commercial sources at the Blood Center. Finally, they also obtained a variety of small antibody testing kits of the sort that might be sold to the general public.
For the sophisticated tests, the results were generally good. Out of a population of 370 individuals who had their infection confirmed while active, the ELISA test done in the labs showed that over 96 percent had detectable antibodies to one of the viral proteins. Testing against two other viral proteins showed 85 and 89 percent of the population had antibodies. Only about 2 percent had no detectable antibodies using this assay.
Things were good when the high-throughput diagnostic machines were used to run tests. These picked up antibodies in 91 and 96 percent of the population, depending on the manufacturer. The small commercial tests, however, didn’t do especially well, with less than 80 percent of them correctly identifying the presence of antibodies.
The researchers did perform a small number of tests on negative controls—samples taken from donors prior to when SARS-CoV-2 reached the United States. While these were enough to show that the tests were working properly, they didn’t look at enough negative control samples to determine whether any of the tests had a high frequency of false positives.
Variations
As we mentioned above, the more sophisticated assays used here don’t just detect the presence or absence of antibodies—they provide an estimate of how much is present. And here, the researchers found that the quantities varied widely. The difference between the highest and lowest values produced by the test varied by a factor of over 1,000.
It’s difficult to know whether that means that the higher immune responses are more effective, however. Some antibodies are what is termed “neutralizing,” in that they bind to the virus in a way that keeps it from infecting cells. So, the researchers engineered the coronavirus’ spike protein into an unrelated virus and used that to infect cultured cells in the presence of antibodies from all these donors. This provided a measure of how often neutralizing antibodies were present.
The vast majority (between 80 and 90 percent) of the donors had some level of neutralizing antibody. But the levels of neutralization went up roughly in line with the amount of antibody present—which, as we’ve just discussed, varies considerably. The end result is that the levels of neutralizing antibody varied over a range of 40,000-fold differences. Unfortunately, this means that, in all likelihood, the variability of the antibodies we see roughly correlates with the amount of protection they’re likely to offer. (Neutralizing antibodies haven’t yet been demonstrated to be effective in patients.)
Waiting for the herd
While this is a draft paper and should be viewed cautiously, the results are largely in line with a separate draft study of 175 patients done by an unrelated group of researchers in China. This also saw high levels of variability and a small subset of patients that had no detectable antibodies to SARS-CoV-2. Another group based in New York found even higher levels of variability, but it found neutralizing antibodies in most patients, even if they were rare.
There have also been a number of draft studies that suggest that the levels of antibody produced correlate roughly with the severity of COVID-19 symptoms that a person experienced.
For now, what do we conclude from all of this? The first is that, to get a more accurate picture of who has actually experienced a SARS-CoV-2 infection, we’re going to want to rely on high-throughput assays. Right now, the smaller personal kits may be helpful for people who are interested in getting a rough picture of their own medical history, but the results aren’t up to the quality we’d want for understanding the public’s exposure.
The second thing is that SARS-CoV-2 infection isn’t necessarily producing a robust immunity. We don’t really know what levels of neutralizing antibodies are actually protective, but it’s clear that a lot of people don’t produce many of them after an infection. Which means any plans for generating herd immunity by allowing a controlled level of infection have to be viewed with extreme skepticism at this point. And vaccine developers will need to ensure that the injections produce a consistently high-level response that includes neutralizing antibodies.