In need of speed

Since the coronavirus pandemic began, much of the national media’s attention has focused on newly released medical research to help the country understand the new virus: What’s the best way to manage patients in the hospital? What are possible treatments for COVID-19? And of course, how can we develop a vaccine to protect people from the virus?

Some aspects of the fight progressed faster than many expected. Operation Warp Speed succeeded in bringing vaccines to market both quickly and safely. Yet other research developments seemed to lag, coming in weeks or even months later than planned. The University of Oxford’s STOIC trial, which discovered that the generic steroid budesonide helps against COVID-19, had planned to enroll its last patient in mid-November of last year—but the study ultimately needed almost two months more. What was the holdup?

To understand that, let’s consider how to design a research study to yield useful information. COVID-19 presents unique challenges to researchers because most patients recover: Its 99 percent survival rate means researchers would expect to need 100 patients to identify one COVID-19 death. Studies focusing on other outcomes—improving severe symptoms, for example—face similar challenges.

Quality research studies vary in their details, but each one builds on this basic framework:

• It asks a specific question (testing a hypothesis).
• It includes a group that doesn’t get the proposed treatment (the control group).
• It follows a large enough group of patients, for a long enough period of time, to detect whether the study group or the control group fares better (it has sufficient power).
• It assigns patients to the study group or control group randomly, so that the groups will generally be comparable (it ensures a representative sample).
• It considers whether the proposed intervention helped and whether other factors might interfere with knowing that (it analyzes its results).
• Other researchers, independent from those who did the study, double-check the work (the peer review process).

As with vaccine development, research studies can be sped up—but only so much before quality suffers. Researchers carefully calculate how many patients they’ll need to enroll in a study in order to achieve accurate results, but they often struggle to find those patients quickly after diagnosis due to legal barriers protecting personal health data.

Staff at research journals last year worked hard to minimize delays with peer review and editing: The time between a study’s submission and its publication dropped by almost 50 percent during the pandemic. Yet it still averaged about 60 days at some of the top journals amid the unprecedented rush of journal submissions in 2020. A Lancet article reported in May that the Journal of the American Medical Association received 11,000 submissions in the first five months of 2020, almost three times more than the same period in 2019. This makes the STOIC trial’s path to publication all the more impressive: Its results reached the public less than one month after the last patient enrolled.

The Lancet article also pointed out that in the research world’s well-intentioned haste to do something about the pandemic, it ended up with many small, poorly designed studies, precluding any meaningful attempt at combining their data into larger, stronger studies. Those initial studies did serve as a source of hypotheses for better-quality research, but they also confused the public. For instance, the presence of “research” supporting drugs like hydroxychloroquine that were later shown to be ineffective against COVID-19 stoked online conspiracy theories and social-media debates.

The Lancet authors argued that research teams would have served the public more effectively by coordinating their efforts toward fewer, but higher-quality, studies—and that they would have ultimately saved time, and lives, by doing so.

Still, perhaps the greatest surprise about the pace of research during the COVID-19 pandemic was not how slowly it moved, but rather how quickly, given the year’s chaos. In reflecting on the good and the bad of our response to the pandemic so far, let’s remember the hard work of those who strove to guide it with quality data.