Is the peer review system sustainable?

What do Godzilla and scientific peer review have in common? Each by rights ought to be collapsing under its own weight. Yet somehow they stand.

The good news — for movie fans and scholars alike — is that the behemoths appear to be stable for the moment. But, in the case of peer review, at least, it wouldn’t take much for the system to become perilously imbalanced.

As it stands now, according to a new study, the pool of peer reviewers is able to keep up with the massive number of new papers published each year in biomedicine — more than 1 million, and climbing.

The amount of time reviewers spend on the task is, simply put, gargantuan: an estimated 63.4 million hours in 2015 alone, according to the authors of the study, or roughly 7,300 years’ worth of reviewing. If journals were to compensate those reviewers at a reasonable rate of $75 per hour, that’s on the order of $4.5 billion of labor.

But the problem is, the distribution of labor is highly uneven. The researchers found that 5 percent of reviewers were responsible for nearly 30 percent of those hours. And that could lead to shoddier reviews, they wrote: “These ‘peer-review heroes’ may be overworked, with risk of downgraded peer-review standards.”

The paper, published in PLOS ONE, included a few other interesting nuggets about the system. Researchers in the United States, for example, punch above their weight, reviewing a greater share of the world’s papers than they publish. On the other hand, Chinese scientists are net importers of reviewers’ time, publishing more globally than they review — twice as much, in fact. A general rule of thumb that many scientists use is to review one paper for every one they submit.

As we and others have argued, peer review is a deeply flawed system, but one that deserves fixing, not scrapping. The latest study does nothing to change that view. It does, however, point to a few simple changes that could go a long way toward shoring up the structure.

The first is paying reviewers for their time. Although the risk is that the “peer-review heroes” might get a windfall, it’s likely that many scientists who would like to review but feel they can’t spare the time will decide a few extra bucks is worth the extra work.

That could bring about a class of professional reviewers — recently retired academics, perhaps, who bring a wealth of experience to the task, or consultants like Jonas Ranstam, who recently won an award as the world’s most prolific reviewer.

Publishing fewer papers would be another means of rebalancing the load, as Tricia Serio, a biologist at the University of Arizona, argued in a post on the Conversation. “The first step is to reset and clearly state our standards for quality in both publishing and peer reviewing,” she wrote. “The outcome will certainly be fewer publications in biomedicine, but their individual impact will be greater.”

Perhaps. But another solution would be to open the spigot for more and different forms of peer review. Sites such as PubPeer.com and PubMed Commons allow for peer reviewers to participate in ways that are less time-consuming but, in aggregate, far more valuable than the anonymous pre-publication process that happens these days. The old guard has resisted that trend, just as those who may not live until climate change has disastrous effects might ignore those realities, too. But they’re not the ones who will have to turn down the coming tsunami of peer review requests.