In this pandemic, we’re swimming in statistics, trends, models, projections, infection rates, death tolls. Nosek has professional expertise in interpreting data, but even he is struggling to make sense of the numbers.
“What’s crazy is, we’re three months in, and we’re still not able to calibrate our risk management. It’s a mess,” said Nosek, who runs the Center for Open Science, which advocates for transparency in research. “Tell me what to do! Please!”
Scientists are still trying to understand the virus they call SARS-CoV-2, which causes the disease covid-19. Basic questions are not fully answered: How deadly is this virus? How contagious? Are there different strains with different clinical outcomes? Why does SARS-CoV-2 create a devastating disease in some people while leaving others without symptoms or even knowledge that they were infected?
With stay-at-home orders expiring and businesses reopening, all the scientific data is being scrutinized anew. But the numbers are often ambiguous, with large margins of error. And because this is still an early phase of the pandemic, scientific findings have to be couched in tentative, provisional, sometimes squishy language that is festooned with caveats and admitted limitations.
The experts shy away from predictions and instead offer “scenarios.” For example, last week the Centers for Disease Control and Prevention published a document titled COVID-19 Pandemic Planning Scenarios that offered guidance to public health officials. The document gave a wide range of numerical estimates for the contagiousness and lethality of covid-19. The guidance was presented with a cautionary preamble: “Information about [covid-19’s] biological and epidemiological characteristics remain limited, and uncertainty remains around nearly all parameter values.”
The CDC added that the numbers presented are “not” — the word is boldfaced for emphasis — “predictions of the expected effects of COVID-19.”
With the science fuzzy, people are forced to do their own calculations and estimates, and figure out what’s safe and what isn’t. They have to decide whether to go to religious services, or head to the beach, or take a summer road trip.
One of the fundamental problems is that the virus is stealthy, with a time delay of about six days on average between infection and symptoms. A sick person may delay getting tested or going to the hospital. The official covid-19 numbers typically lag behind the on-the-ground reality. Lurking out there is a possible second wave of infection, and the danger is that the wave will be detected only when it’s about to crest.
Even a trained eye can have trouble bringing the pandemic into sharp focus and knowing what to do.
“I wish I had any confidence at all in any of the prognostications,” Nosek said.
The inescapable facts
The ideologically divisive media ecosystem does not help in a crisis like this. Distrust of the numbers, and of institutions that rely on professional expertise, is widespread. Science has been dragged into partisan politics and the culture wars, and special interests cherry-pick the data to advance their arguments.
Some critics of the stay-at-home orders have declared the numbers to be fake or exaggerated by scientists or the news media. There are multitudes who subscribe to the theory that the pandemic is fundamentally a hoax designed to hurt the reelection prospects of President Trump.
But the spike in deaths in recent weeks is not a mirage and can’t be ascribed to misdiagnosis. The CDC tracks “excess deaths,” from any cause. The agency last Wednesday came out with its latest estimate: 84,891 to 113,138 excess deaths in the United States since Feb. 1. The agency noted that mortality data from recent weeks remains incomplete.
That death toll does not specify the excess deaths as covid-19 deaths. The numbers might include a significant number of people who did not go to a hospital, or seek some other form of care, for fear of catching the virus. Nor does it break out the decrease in traffic deaths, or other causes of death that might be affected by the shutdown.
What the CDC estimate does is confirm the scale of the national crisis. The inescapable fact is that roughly 100,000 additional deaths occurred in the United States in a matter of weeks, and did so despite a massive society-wide adoption of social distancing and other preventive measures. Without that emergency action and public response, the death toll surely would have been much higher.
The question of the true lethality of the virus remains the subject of controversy. When the CDC put out its guidance last week, it estimated that 0.2 to 1 percent of people who become infected and symptomatic will die. The agency offered a “current best estimate” of 0.4 percent. The agency also gave a best estimate that 35 percent of people infected never develop symptoms. Those numbers when put together would produce an “infection fatality rate” of 0.26, which is lower than many of the estimates produced by scientists and modelers to date.
If the severity of covid-19 has been significantly overestimated, and further research confirms this, critics of the national shutdown will cite this as evidence that the country overreacted to a virus that is not that much worse than seasonal influenza.
The brutality of the virus weakens such a purely statistical argument. Young people are usually spared severe illnesses, but among people of advanced age, this is a disease that can strike down a vigorous person quickly and cruelly, often leading to an isolated death with family members unable to be at the bedside. And tens of millions of people in the United States are at elevated risk because of chronic underlying conditions, such as diabetes and hypertension. These people do not care so much about statistical averages as they do about their personal risk level.
Even the low estimates must be viewed in the context of a population that, at the start of this year, was completely susceptible to this highly contagious virus. Fatality rates fluctuate over time and from place to place, and depend on many variables, including demographics and access to health care. But if the CDC’s “best estimate” is correct and remains constant, and half the people in the United States become infected during the next two years without the introduction of a proven drug treatment or vaccine, that would result in about 426,000 deaths.
The lethality of the virus has been hard to estimate because of the lack of testing and the paucity of solid data on how many people have been infected. That data is now coming in, however, including a report by researchers at the University of Southern California and the Los Angeles County health department, published in JAMA, that described a survey of Los Angeles County residents who were tested for antibodies to the virus. The authors estimated that about 4 percent of the population had been infected as of April 10 and 11.
Although the report did not offer an infection fatality rate, lead author Neeraj Sood, a professor of health policy at USC, said it would probably be 0.13 percent for people outside nursing homes and 0.26 percent — identical to the CDC best estimate — when people in nursing homes were included.
No one in the survey lives in a nursing home. All of the volunteers tested were “community-dwelling individuals,” he said.
“Depending on how you make the assumptions, you can get different answers for the infection fatality rate,” he said.
The published report, in keeping with scientific norms, acknowledges that it “has limitations” and states them: “Selection bias is likely. The estimated prevalence may be biased due to nonresponse or that symptomatic persons may have been more likely to participate. Prevalence estimates could change with new information on the accuracy of test kits used. Also, the study was limited to 1 county.”
‘Projections are not forecasts’
The pandemic has cast a bright light on epidemiology and the computer models that played a key role in spurring the massive shutdowns in March. Some critics of the models have called them flawed and incorrect. A favorite target of right-wing news sites has been an Imperial College model that in late March said 2.2 million people in the United States could eventually die if the country took no measures to halt viral transmission.
The White House coronavirus task force relied on that model and a number of others, including one from the University of Washington. When Trump extended the closure recommendations by 30 days on March 31, members of his task force revealed a forecast, based on many models. Even with social distancing and other mitigations, it showed, the coronavirus would kill between 100,000 and 240,000 people over some unspecified period of time.
That number was stunning. The death toll was still less than 4,000 nationally.
The forecast now looks well-founded, with hundreds of deaths added daily and a vaccine or reliable therapeutic treatment still probably far in the future.
Scientists like to say that all models are wrong but some are useful. The decision by Trump to order an initial 15-day national shutdown came March 16, when the country had reported only 85 deaths, according to a Washington Post tally. Models — however imperfect and dependent upon assumptions that might be incorrect — gave political leaders and the public an accurate sense of the likely scale of the covid-19 epidemic facing the nation.
“Forecasts can be wrong, but the likelihood is that they’re more likely to be right than wrong in some of these locations. They’re a warning sign that this might be a time when you should modify your behavior,” said David Rubin, director of PolicyLab at Children’s Hospital of Philadelphia, referring to a model his center has produced that shows the counties in the United States most vulnerable to new waves of infection.
“We don’t want to be purveyors of doom,” said Samir Bhatt, a geostatistics expert at Imperial College in London who created a model looking at how reopening the economy could play out across the United States. “We just wanted to get it right. Projections are not forecasts. They’re just scenarios. They’re helping us understand things in the absence of data. No one can really predict the future.”
One common misperception today is that scientists oppose the reopening of the economy. Anthony S. Fauci, the director of the National Institute of Allergy and Infectious Diseases, said in a television interview Friday that if stay-at-home orders were imposed for long, it would cause “irreparable damage,” but he urged people to “please proceed with caution.”
Jeffrey Shaman, an influential epidemiologist at Columbia University, said that the health of the economy matters, and that the nation needs to restore economic activity in a way that keeps people safe.
“We have to do both those things,” Shaman said. “We want a functioning economy, and we don’t want people getting sick.”
Craving quick answers
Getting the science right is critical not only because of this ongoing health emergency but also because it’s going to happen again. This is a natural event — zoonosis, the transmission of a disease from an animal to a human — made more common by the burgeoning human population, exploitation and illegal trafficking of wildlife, and the globalization of commerce that has turned the planet into a mixing bowl.
Science takes time. By tradition, it is a deliberate endeavor, and not designed for emergencies, quick answers and certainty, however much that may be craved.
Not only do experiments take time to devise, execute and analyze but the broader scientific community usually has a chance to scrutinize and potentially falsify the results. This process has been largely abandoned during the pandemic.
Reports that haven’t been peer-reviewed are posted online, on what are called preprint servers, and often generate headlines before they’ve been pressed through the filter of outside examination. Much of what is received wisdom about the virus today could be significantly revised as research continues.
Ilhem Messaoudi, an epidemiologist at the University of California at Irvine, said in an email that, in this age of preprints and social media, many of the nuances of research have been lost in the public discussion. “50 character headlines have prevailed over careful and reserved discussions,” she wrote. “With a global pandemic such as this, the public wants the scientific leaders to speak with certainty, but that is not possible.”
She added, “Fear also drives people to think in binary terms such as you are for public health or for the economy; you are for masks or you are reckless.”
Nosek, the U-Va. professor, said that “the pandemic has exposed the messiness of science.”
That’s how science always is, he said — but we don’t usually see that truth exposed so vividly.
“We all want answers today, and science is not going to give them,” Nosek said. “Science is uncertainty. And the pace of uncertainty reduction in science is way slower than the pace of a pandemic.”
