T

he results of bioscience research are among the great achievements of the modern world. From them have come deep insights into the fundamental nature of life and effective means to prevent and treat disease, with many more on the way. But flaws in peer review, a critical step in communicating science, hamper its progress.

Communicating the results of basic science, clinical trials, observational studies, case reports, and the like is essential to the forward movement of science. The peer review system is at the core of this process. It works like this: a journal editor asks outside experts to critique a manuscript and assess its suitability for publication, with or without modifications. Editors use these peer reviews to help determine whether or not to publish a report.

The peer review process has existed for more than 150 years. It became routine in its current form after World War II, as funding for bioscience research exploded along with specialization and professionalization in the bioscience community.

advertisement

In earlier times, editors evaluated manuscripts submitted for publication and rendered decisions themselves, asking for informal consultations as they saw fit. Today, the scope and diversity of science demands input from expert reviewers to enhance the quality and reliability of published reports.

Two common features of the current peer review system subvert the goals of science, and should be changed: The product of peer reviews generally can’t be seen by the scientific community, and reviewers are almost always anonymous. Although the vast majority of scientists living today have known only this approach, there is no compelling reason to continue using it and many reasons to revise it.

Today, the only people who are privy to the results of a peer review are the authors and the journal’s editors. I strongly believe such reviews should be part of the public record of science, just as the final published reports are. Why? There is a huge variation in the quality of reviews, from deep and insightful to shallow and misinformed. When peer review is cloaked in secrecy, there are limited incentives for performing high-quality reviews. That allows bias, carelessness, conflict of interest, and other deficiencies to persist without a way to penalize those who generate inadequate reviews.

Reading a paper in “final form” without access to the back-and-forth of reviews and editorial commentary deprives readers of key insights into how the research was conducted. Instead, we see a tidy but inaccurate portrayal that misrepresents the messy path that research often takes in going from the lab or clinic to publication. As a consequence, our ability to properly educate scientific trainees suffers, and our capacity to provide an accurate history of discovery is impaired.

Finally, public access to peer reviews would make it possible to examine the factors that influence the quality of peer review and editorial decision-making. With hidden reviews, such research is impossible, limiting needed improvements to the process.

Are there advantages to hidden reviews that counterbalance these objections? In an earlier era of print-only medical journals, space was limited by cost. Publishing peer reviews would have been expensive. Today, with online publishing essentially universal and the cost of posting a peer reviewer’s comments negligible, that objection disappears. Apart from this, I can think of no other advantage to keeping peer reviews private — yet tradition and inertia sustain this practice.

What about revealing reviewer’s identities? Though this proposal is more controversial, there are powerful reasons to support it as well. Just as reviewer anonymity provides incentives for shoddy or inappropriate reviews, reviewer identification discourages such work.

Providing a solid peer review takes time and energy, and reviewers receive no financial compensation for their work. Identifying reviewers would at least make it possible to recognize the quality and impact of reviews when faculty members are evaluated for promotion. Consider a reviewer whose critiques and suggestions enabled authors to transform their submission from an inferior report to an important contribution. Shouldn’t the reviewer be credited with that real and documentable contribution? That’s not possible today.

And just as with making reviews public, making reviewers public would enhance research into the factors contributing to the quality of reviews.

Over the past 30 years, peer review reformers have argued for these and other changes. One controlled experiment involving the BMJ showed that when properly alerted, reviewers accepted the loss of anonymity and produced reviews of undiminished quality. Quite a few other journals are testing the waters, with promising results. But the vast majority of journals continue to employ hidden and anonymous peer review and show no inclination to give it up.

Despite the evident advantages of being more transparent about peer review, some editors believe that their editorial oversight compensates for deficiencies in the quality of peer reviews. That may be true for some journals, but it is far from universal and it’s impossible to independently evaluate.

Instituting the two changes I have described might entail added work and cost that journal editors would rather avoid. Some scientists might also resist these changes. A prominent reason for their resistance is hardly complimentary to the scientific community: Some reviewers, especially junior scientists, fear that critical but honest reviews of work by senior scientists in the field would subject them to retribution by these senior figures, who might someday be asked to review their grants, papers, or promotions. Faced with such fears, they might decline requests to do peer reviews, or provide less honest reviews seeking to pander to the authors.

This isn’t just a hypothetical concern — I have heard it from many faculty members at Harvard Medical School, as well as from some journal editors. To the extent it is real, this brings shame on a profession committed at its core to the pursuit of knowledge and truth. Rather than accepting this outcome as unavoidable and countering with secrecy and its adverse consequences, this problem must be addressed through a variety of concerted approaches.

Leaders in biomedicine should take seriously the need to improve the processes we use to evaluate and publish science, which are now hobbled by lack of transparency. Concerns about the irreproducibility of bioscience research suggest that many other aspects of the research ecosystem need improvement, from how we educate scientists in research methodology and ethics to how we apportion research funding and the criteria we employ for appointment, promotion, and recognition.

But our approach to journal peer review is especially ripe for change. I urge the science publishing industry, the academic community whose research sustains it, and the organizations that fund it to cooperate in considering and responding constructively to these issues.

Jeffrey S. Flier, MD is the Harvard University Distinguished Service Professor, the Higginson Professor of Physiology and Medicine, and the former dean of Harvard Medical School.

Sign up for Morning Rounds

A daily dose of health and medicine news — and a finalist for Digiday’s best email newsletter.

Recommended Stories