Foundations of scientific sand

When a scientific paper is retracted, researchers should stop using that debunked research for their own studies. But according to University of Illinois information sciences professor Jodi Schneider, that’s not happening nearly enough.

Schneider’s research, published in the journal Scientometrics, indicates scientists continue citing research studies even after those studies have been withdrawn. The finding gives lie to the hope that digitally published research journals would make scientific literature more accurate.

“The current information environment facilitates the spread of research papers,” Schneider wrote. “But basic facts about these papers, such as their retraction status, do not spread as swiftly as the PDFs or citations to these papers themselves.” In other words, lies move faster than the truth.

In 2005, the pulmonology and cardiology journal Chest published a study claiming omega-3 fatty acids helped reduce inflammatory markers in patients suffering from chronic obstructive pulmonary disease. Three years later, the journal published a retraction admitting the study’s authors had falsified clinical trial data.

In theory, that should have been enough to erase the study from the medical literature moving forward. But Schneider’s study found 144 papers that cited the omega-3 study—112 of them published after Chest pulled the article in 2008. Of those direct citations to the discredited study, 96 percent make no mention of the falsified research. Schneider found even more problems in the second-generation research. More than 2,500 studies cited one of the 144 papers that used the omega-3 article.

According to Schneider and her co-authors, journal editors don’t closely check the bibliographies of submitted papers. “It’s really rare for any journal to be scanning reference lists,” Schneider told the Illinois News Bureau.

More than 12 years after Chest published its retraction of the omega-3 article, the journal’s website includes no notice on the study’s page.

Removing bad science from the academic literature proved more difficult before the adoption of electronic databases. Rescinded papers could sit for years in physical libraries without any sort of notice attached. Researchers hoping to keep up had to scour scientific publications for notices. A 2010 study by Finnish psychologist Kalevi Korpela found a psychology article formally retracted in 1982 still being cited in 2006.

In the electronic age, some online databases make disclaimers easier to see than others. PubMed, a National Institutes of Health research database focusing on medical and life science papers, uses a large red bar with the words “retracted article” atop studies that have been pulled. Other databases are less clear. “Many databases and search engines, such as Web of Science, Google Scholar and Scopus provide nearly no warning of the retraction in their search results,” Schneider said.

Another problem Schneider identified was the sheer volume of published material. Only four out of 10,000 publications are eventually retracted, she said. The overall number “is really, really tiny,” she told the Illinois News Bureau. “It’s a regular occurrence that things get retracted, but people publish so much.”