States that adopted vastly different approaches toward the pandemic—such as Texas, which reopened "aggressively" last spring, and California, which is reopening slowly—appear to have similar rates of Covid-19 cases and fatalities, spurring questions about the efficacy of various prevention measures. Here's how experts are interpreting the data, according to Vox's Dylan Scott.

The Covid-19 resources you need right now

'A maddingly difficult question'

According to Scott, determining which "interventions actually work is one of the most important lessons the United States could learn from the Covid-19 pandemic" because it will enable the country to respond more effectively when the next pandemic occurs.

The 6 biggest Covid-related myths we've seen, busted

However, making those determinations is "a maddeningly difficult question to answer," Scott writes, because the pandemic itself was constantly evolving, because states adopted—and rolled back—different policies at different stages, and because the virus over time became increasingly politized.  

"State policies mattered," Jen Kates, director of global health policy at the Kaiser Family Foundation, said. "But it is hard to know and we may never know whether policies made at the state level were able to overcome all the other challenges of this pandemic."

But although that state-level variation contributed to the "policy morass" that hindered the nation's response to the pandemic, Scott writes, the differences in various states' responses do provide some insight into whether "certain interventions were more effective than others."

Masking requirements had a clear benefit, experts say

For instance, according to Scott, the "science on masks is clear: They work."

"Indoor masking guidance was proven to be effective," Amesh Adalja, senior scholar at the John Hopkins Center for Health Security, said. "When you look at it all, I think that is probably going to be the one that shows the most effect. ... Most things can be done safely if people socially distance and wear a mask indoors in an unvaccinated setting."

According to Scott, currently available research "supports that conclusion," including a March 2021 CDC study that found mask "mandates were associated with slower [coronavirus] transmission" based on county-level data and a June 2020 study that estimated mask mandates "averted some 200,000 Covid-19 cases by mid-May" 2020.

Is America's coronavirus future 'good,' 'bad,' or 'ugly'? It's all three.

But some have questioned why, if masking was so effective, the "dire warnings about what would happen when Texas lifted its mask mandate for good in March 2021 never materialized," Scott writes.

To that, he points out that the masking rollback "took place in a very different context from the spring of 2020." Not only did more people have some level of immunity to the virus via prior infection or vaccination, Scott writes, but—because of how political the pandemic had become by that point—a state mandate was unlikely to change individual's behavior about masking or not masking; people were going to keep on wearing a mask or keep on not wearing a mask no matter what the government advised.

The efficacy of social distancing, lockdowns is mixed, experts say

According to Scott, if you assess states by the number of Covid-19-related deaths per capita, "it's hard to discern much of a pattern" when it comes to measures that involved social distancing, such as lockdowns and stay-at-home measures.

For instance, several states in the Northeast, such as New York and New Jersey, where the virus arrived early in the pandemic, were "most aggressive in restricting people's activities" and they "rank at the top." But the same is true for states such as Arizona and Mississippi, where the virus arrived later in the course of the pandemic and where state leaders maintained comparatively "relaxed … policy responses."

Part of the problem is the coronavirus' "different timelines" throughout the country, Scott writes. In early March and April 2020, when 40 states issued stay-at-home orders, Covid-19 was surging mainly in New York—it had "hardly touched the Midwest or the South." As a result, according to Scott, states in those regions got "a false sense of security," with many lifting their stay-at-home orders before "their own outbreaks really got going"—and some, like Florida, reestablished partial restrictions only when the virus did take off locally.

Moreover, since nearly all states had "relaxed some restrictions by May or June 2020," the "true lockdown period in the U.S. was actually quite brief, and as states were reopening in the summer, the virus had more opportunities to spread," Scott writes.

'The Never Again Plan': Moderna CEO Stéphane Bancel wants to stop the next Covid-19—before it happens

In addition, according to Scott, public support for these measures in many places weakened as the pandemic progressed, so even when states reinstituted some partial restrictions later on, many people may have simply have not adhered to them. "During the remainder of 2020, had such measures remained in place, they would have impacted the spread of Covid-19," Aaron Yelowitz, an economist at the University of Kentucky, said. "But not nearly as much due to lower 'buy-in' from the public and increased polarization."

Another complicating factor, Scott writes, was how many states imposed a raft of different types of closures and restrictions at the same time, making it hard to identify which one measure—if any—was the most effective. For instance, although a recent study in Nature concluded that social distancing policies in aggregate appear to have slowed the pandemic, it noted, "Given that most states enacted multiple policies to encourage social distancing over a short time period, it is not possible to estimate the independent effects of individual policies."

And other research suggests that certain types of lockdown measures were more effective than others. For instance, a July 2020 study found that while stay-at-home orders, bar closures, and restaurant closures appeared to slow the spread of Covid-19, the information about school closures and prohibitions against large gatherings was less clear. Two more recent studies came to the same conclusion, Scott writes.

A missed opportunity

Although there "was likely no policy that would have eradicated the virus entirely" because of how quickly it spread, the United States failed to leverage the limited opportunity "it had created for itself through the lockdowns," Scott writes.

Specifically, even if the virus could not be eradicated, Scott writes, the country could have established a testing, tracing, and isolation system for Covid-19 that would have prevented small outbreaks from turning into widespread surges—the sort of system used "in global Covid-19 success stories like South Korea." But America "failed to scale up its testing or contact tracing capabilities," Scott said, which means as states rolled back their restrictions, "America was ill-suited to track it" and the "window that these early lockdowns had briefly opened was closed" (Scott, Vox, 6/3).