Is that medical treatment really effective?

Physicians prescribe medications and treatments to their patients based on research results that say those methods work. And patients follow their doctors’ recommendations because they believe the research is accurate. But a new study suggests test results from older research might not be as reliable as previously thought.

The researchers looked at large-scale medical studies conducted by the National Heart, Lung, and Blood Institute (NHLBI) between 1970 and 2012. In 2000, the NHLBI started requiring researchers to preregister their studies. Before that, 57 percent of human research trials showed effective treatment results. But after the organization put preregistration requirements in place, studies indicating effective treatments almost vanished. The number of studies with positive results plummeted to a mere 8 percent.

The researchers looked at several possible explanations for their findings—the use of placebos, industry financing of research, better trial management, and fewer new treatments being developed in recent years. But they concluded the lack of preregistration requirements and guidelines to encourage research transparency prior to 2000 likely led to many false positive results.

“A growing collection of trials suggests that promising treatments do not match their potential when systematically tested and transparently reported,” the researchers said.

In 1997, the Food and Drug Administration passed the Modernization Act, which created the ClinicalTrials.gov registry and required medical researchers to register any scientific human trial study before they start gathering data.

The act requires researchers to provide the purpose of the study, information on how they will recruit participants, the research design they will use, the criteria for participant eligibility, the locations where the study will be conducted and the primary and secondary outcomes for which they are testing. Identifying the primary and secondary outcomes the researchers are looking for is especially important because it prohibits researches from tweaking the results or picking and choosing only statistical outcomes that show a positive result.

“It’s more difficult for investigators to selectively report some outcomes and exclude others," Veronica Irvin, coauthor of the study and a health scientist at Oregon State University in Corvallis told the journalNature.

In addition to preregistration requirements, a group of medical experts introduced The Consolidated Standards of Reporting Trials (CONSORT) in 1996 and expanded it in 2001 to require greater transparency from researchers in reporting randomized clinical trials. Many major journals began to require registration of clinical trials as a condition for publication, and in 2004 the International Committee of Medical Journal Editors started to require CONSORT reporting in all major journals.

Despite current registration requirements, problems remain with ensuring the ClinicalTrials.gov record and the published research results concur, according to theAnnals of Internal Medicine.

This study should be a wake-up call, neurologist Steven Novella of Yale University wrote on his NeuroLogica Blog. While it may be disconcerting to think of the time and money wasted on treatments based on false positive results, not to mention the consequences for patient health, it is reassuring to know the guidelines requiring preregistration and transparency appear to work, he wrote.

“At this point there is sufficient evidence to warrant full requirement for all human research to be registered prior to collecting data, declaring methods and outcomes to be measured,” he concluded. “We need high standards of scientific rigor with full transparency in reporting.”