Placing trust in science can be easier when findings are confirmed, but a new survey finds that most scientists believe there is a “crisis” in reproducibility.
Specifically, 52 percent reported that replicating results is a “significant” problem and another 38 percent believe a “slight crisis” exists. More than 70 percent of researchers have tried and failed to reproduce another scientist’s experiments, according to Nature, which canvassed 1,576 researchers. And more than half of the respondents reported that they failed to reproduce their own experiments.
Yet one-third believe that failing to reproduce results means that a study is probably just incorrect, and most of those asked say that they continue to trust published findings. Moreover, 73 percent think that at least half of the papers in their own fields can be believed, with physicists and chemists expressing the most confidence.
Nature wrote that researchers often assume there is usually a “valid reason” for this belief, and the perception is compounded by a lack of incentive to publish positive replications. Moreover, medical journals may be reluctant to publish negative findings, according to Nature.
The survey presents a confusing picture at a time when there is growing importance attached to reproducing results.
For instance, a Merck executive recently suggested that drug makers should be entitled to get their money back for potential treatments licensed from universities if the company is unable to reproduce the results in subsequent experiments. In his view, the inability to replicate findings is reason for altering the nature of the relationship between academia and industry.
At the same time, the pharmaceutical industry has been under pressure to release trial results in order to verify claims about their medicines. The issue accelerated in the wake of several scandals about undisclosed side effects. Last year, a group of researchers analyzed trial data for a GlaxoSmithKline (GSK) antidepressant and found the original safety claims could be not reproduced.
Turning back to the lab, the survey found that scientists cited several reasons for difficulties reproducing results. More than 60 percent pointed to the pressure to publish and selective reporting as regular issues. More than half cited insufficient replication in the lab, poor oversight, or low statistical power, Nature wrote. And some said the use of specialized techniques, for example, can be difficult to repeat.
To improve outcomes, the scientists said that better study designs and better statistics are needed, Nature wrote, adding that one-third of the respondents said their labs had taken steps in the past five years to bolster the odds of replacing results.