Mathematics in everyday life

How to improve the peer review process

Advertisements

If you have published in an academic journal, I am sure you have been amazed by the inefficiency of the process. A new article by Dwight Kravitz and Chris Baker in the online journal Frontiers in Computational Neuroscience presents a good overview of what is wrong with the current system, and offers interesting suggestion of how to improve it.  Here is a short summary:

Our current peer review system has been in place for centuries. In the past, producing an online journal article was quite expensive. To limit cost and maximize readership it was imperative that the publisher only print a small number of the very best articles. Having candidate articles reviewed by a panel of experts assured both quality and selectivity.

However, while print journals are still deposited in libraries, I would estimate that more than 95% of the time scientists access articles online. In the present environment the traditional peer review process has become very inefficient. In most disciplines, authors will send an article first to a top journal in the field. If it is rejected there, the article will be sent to a journal further down the list, and so on, until the article is eventually accepted and published. Based on polling a number of colleagues, Kravitz and Baker estimate that it takes on average 2.6 revisions and 6.3 reviews to get an manuscript published. Since the article will eventually appear anyway, more than half of these reviews are not necessary. Moreover, a lot of effort in the reviewing process goes into determining whether the manuscript contains results that are sufficiently novel, and whether it is appropriate for a particular journal. This has little bearing on the scientific content of the article – it should be the scientific community that determines how interesting the results are, and not the two or three umotivated reviewers chosen somewhat arbitrarily by the editor.

In short, nearly every manuscript eventually gets published. The review process does provide valuable feedback to the authors. But much of the process is repetitive and does not improve the quality of the eventual publication.

So what to do? Kravitz and Baker provide an interesting alternative: Let’s keep the useful part of the reviewing process. After an article is submitted, a panel of reviewers should still be asked to offer suggestions for improvement. However, there should be a single round of review. The authors then revise the article, after which the article is published. The Editorial Board provides expert comments on the article, and perhaps rates its importance. Most importantly, the scientific community is then asked to provide comments on the article, and direct questions to the authors. While the article would not undergo changes after publication (except to correct errors, I suppose), the ongoing discussion could be valuable in clarifying, and maybe even extending the published results.

I see several problems with these suggestions. First, the current system remains in place because there are strong interests keeping it in place. The publishing industry is run largely like a monopoly, and makes billions of dollars. This industry will opposed changes that could impact their profits, or potentially even eliminate the need for traditional journals. We scientists are somewhat complicit here, firstly because change requires effort. Secondly, how will we be evaluated for tenure and promotion when traditional journals are gone?

Perhaps, most importantly, who will bear the cost of these changes, and who will bear the publication costs in the future? Having authors pay the cost of publishing biases what can be published. Well funded labs could have a large impact in the field simply because they could publish more. Many mathematicians receive no external funding – should they be required to pay thousands of dollars out of pocket every time they wish to publish a paper? One solution would be that home institutions pick up the costs, instead of paying for journal subscriptions. However, some good scientists do not have university affiliations.

This summary doesn’t do justice to the ideas of Kravitz and Baker. If you are in academia, I urge you to read their article. Even if you disagree with their suggestions, I think it is important to start this conversation.

Advertisements

Advertisements