New studies suggest huge undercount of coronavirus infections — but are they right?
The number of COVID-19 cases in Los Angeles County may be more than 50 times greater than the official count, according to preliminary results from a new study by the University of Southern California, which estimates that as many as 442,000 adult residents of Los Angeles County may have already been infected.
The implication, as lead USC investigator Neeraj Sood told reporters Monday, is that “the true extent of the infection in our communities” is far greater than previously known.
“We actually found that about 5 percent of Angelenos have in fact, probably, at some time, been infected with COVID-19,” explained Dr. Barbara Ferrer, director of the L.A. County Department of Public Health, which partnered with USC on the study. So now “we are able to understand what the real denominator is” in calculating the rate at which people who are infected develop symptoms, require hospitalization and die.
But should we trust those conclusions?
If accurate, these estimates would fundamentally change our understanding of the coronavirus. Experts have long suspected that real-time testing is catching only a fraction of the total infections. But they have tended to put that fraction somewhere between one in five and one in 20. If in reality we’re identifying only one in 50 infections, or fewer, that would make COVID-19 a lot less deadly than previously believed, while also making it a lot more contagious (and asymptomatic “silent carriers” a lot more widespread). That would be a paradigm shift in how to combat the virus.
The problem, though, is that these studies may not be accurate.
Known as a “seroprevalence survey” or “serosurvey,” the preliminary USC study is the second of its kind to come out in the past few days. The first was conducted by Stanford University researchers in Santa Clara County, Calif. — aka Silicon Valley — and released Friday. Together they represent the first attempts at this kind of research in the U.S.
Both serosurveys worked the same way. Researchers recruited hundreds or thousands of local residents to participate. They tried to ensure their “sample” was representative of the county’s overall population. They tested all participants with the same serology test, which is supposed to reveal who has already been infected by detecting antibodies in their bloodstream — even if they never showed symptoms or were never tested for an ongoing infection by the more typical nasal-swab method. Then the researchers tallied up the total number of positive test results, divided it by the total number of participants and “scaled” that percentage to the population of the entire county to estimate the overall seroprevalence of COVID-19 in the area.
Both studies also produced similar — and similarly startling — numbers. In Los Angeles, the USC team found that 4.1 percent of its 863 study participants tested positive; applied to the county as a whole, and adjusted for a statistical margin of error, that would mean anywhere from 2.8 percent (221,000) to 5.6 percent (442,000) of Angelenos have been infected, according to the USC researchers. (By comparison, 423,000 people had tested positive for COVID-19 nationwide as of April 9, when the study concluded.) Meanwhile, in Santa Clara, the Stanford team tested 3,330 residents and estimated that 2.5 percent (48,000) to 4 percent (81,000) of the county’s population has been infected — or 50 to 85 times the official number of confirmed cases there.
Yet statisticians and infectious disease experts have been poking holes in the Santa Clara study since the second it came out.
One criticism is that the antibody test they used — a kit purchased from Premier Biotech in Minneapolis — has a false positive rate as high as 1.7 percent, according to some estimates, meaning that even if you test only people who have never had the disease, as many as 1.7 percent of them would still test positive.
To a layperson, that sounds relatively low. But when your sample size is small and the disease you’re testing for is rare, it’s actually big enough to render your results potentially meaningless. Of the 3,330 Santa Clara residents tested, 50 came back positive — or 1.5 percent. Most or all of them, in other words, could have theoretically been false positives.
Another possible issue is the way the researchers “weighted” that raw result, adjusting it statistically to reflect the county’s demographics and account for the inaccuracy of the test. This is how they ultimately estimated that between 2.5 percent and 4 percent of residents have been infected. But as experts have noted, such weighting introduces a lot of subjective judgments that can potentially skew the results — especially when the thing you’re trying to measure has a percentage prevalence in the low single digits.
Statistician John Cherian of D.E. Shaw Research, a computational biochemistry company, made his own calculations given the test’s sensitivity and specificity — and conservatively estimated the proportion of truly positive people in the Stanford study to range from 0.2 percent to 2.4 percent of the Santa Clara population. Adjusting for demographics, Cherian’s calculations suggest that county prevalence could plausibly be under 1 percent.
A final possible sticking point is the sample itself. Participants were asked to volunteer via Facebook ads, meaning, as computational biologist and statistician Balaji S. Srinivasan has explained, that the study might have attracted people who thought they were exposed to the virus and wanted to get a test they couldn’t otherwise get — and that those people might have in turn recruited other exposed people. The result could have been a “biased,” nonrandom sample with more COVID-19-positive participants than the population at large.
The Stanford researchers attempted to account for these issues in their study, and on Sunday they said they are planning to release a detailed appendix that addresses the criticisms and incorporates many of these suggestions into the paper itself.
The USC team did not immediately release their methodology, so it’s difficult to compare their findings to Stanford’s. One difference, according to the lead investigator, was recruitment: Instead of using Facebook, USC called and emailed a random assortment of Angelenos from the database of an L.A. market research firm, then set quotas meant to mimic the demographics of the county as a whole. But it’s entirely possible that many of the potential issues with the Santa Clara study apply here as well. After all, USC used the exact same antibody test, the same lab and many of the same overseeing researchers as Stanford — plus the sample size (863) was much smaller. Neither preliminary study has been peer-reviewed yet.
So how can we know the true extent of the coronavirus pandemic? We can’t — at least not right now. The key is to wait until a bunch of different studies — conducted with different technologies and analytic approaches — come out.
That will happen soon enough. The National Institutes of Health in Bethesda, Md., just launched a serosurvey that will collect and analyze blood samples from as many as 10,000 volunteers. A UC Berkeley project, set to begin in May, will test a large and representative swath of 5,000 East Bay residents. UC San Francisco is now testing all 1,680 residents of rural Bolinas and 5,700 residents of San Francisco’s densely populated and largely Latino Mission District for evidence of the virus. And teams in China, Australia, Iceland, Italy and Germany are expected to release the results of their serosurveys in the coming days.
In the meantime, it’s worth keeping the two takeaways touted Monday by USC’s lead investigator in perspective. The first, Sood claimed, is that “the fatality rate is lower than what we thought it would be” — as low as 0.1 percent or 0.2 percent, if both studies are accurate. But again, we don’t know if they are, and we have reason to suspect that their estimates of the total COVID-19 case count are much too high, which would make their estimates of the fatality rate much too low.
But Nood also pointed out that “the fatality rate is not the only number we should focus on,” because “that is not the only number that determines the burden of disease.”
“What the findings [also] show is that only 4 percent of our population has been infected, which means we are very early in the epidemic and many more people in L.A. County could potentially be infected,” he continued. “As that number of infections rises, so will the number of deaths, the number of hospitalizations and the number of ICU admissions.”
And remember: 4 percent or 5 percent is the high-end estimate. The true prevalence of COVID-19 is probably lower. So even in a world where current testing is detecting only one in 50 or so infections, the United States is nowhere near herd immunity. Meanwhile, as the virus spreads, hundreds of thousands or even millions of Americans will never know they were infected — and could be spreading a very contagious disease to other, more vulnerable people.
That may complicate efforts to transition out of lockdown.
“It does for me reinforce the need for everyone to continue to stay home, to observe all of our physical distancing requirements, because there are many, many people throughout the county who are positive for COVID-19 who may not be showing any symptoms,” said Ferrer. “We need to assume that any point in time we could be infected and all of the other people we come in contact with could be infected.”
The bottom line is that whatever the true fatality rate, more than 40,000 Americans have died from the coronavirus. California’s new serology tests suggest that many more will die before the pandemic is over.
Read more:
