Contributing Editor Vlada Limic has been frustrated by the peer review process:

If you are reading this, chances are that peer review is regularly on your mind. It is in fact more than likely that you have recently written or read a peer review report, or discussed some aspect of the editorial process with a colleague.

Scientific publishing is a vast and complex topic, and my intention is to focus on one particular aspect of it only. Before doing anything else, I would like to turn your attention to the discussions by our peers in the IMS Bulletin and elsewhere.

Wikipedia tells us that modern peer review, with anonymous referees included in the process, started only in the 20th century. This is not surprising since, before the expansion of university education, the number of active mathematicians (as an example; pick another scientific discipline if you wish) was minuscule compared to nowadays. The number of math journals was correspondingly small, and the number of papers submitted to each was sufficiently small that the editor-in-chief or the members of the editorial board could read each and every preprint in detail before making a decision. That was the time of artisanal scientific publishing—and I use here the word artisanal with great respect. The changes came first slowly, and then more and more rapidly. I joined the train with LaTeX and the internet already in place. I can therefore only guess at the extent of the transformation that occurred in the early to mid 1990s.

Various problems soon emerged, undoubtedly due to the tremendous and unanticipated simultaneous throughput pressure on all the participants in the editorial process. While problem solving is one of the trademarks of our profession, efficient response to social and demographic changes cannot be done overnight (or by studying a toy model), so our community reacted to these challenges with a certain delay. In particular, volume 39, issue 2 (March 2010) of the IMS Bulletin has a section devoted to various aspects of scientific publishing. Ethical conduct and misconduct (some, but not all, related to peer review) is the theme of the article by Peter Imrey. It is followed by a letter by Dimitris Politis, that offers three interesting thoughts on the editorial process. Just below it, the Bulletin solicits opinions and ideas from other peers.

Written feedback came, at least from Xuming He, Dimitris Politis, and Jean Opsomer in volume 42, issues 2, 3 and 4, respectively. In particular, Dimitris writes in Refereeing and Psychoanalysis, it is “important that the peer review is as fair and unbiased as humanly possible,” and he informs us of the ethical principles about to be adopted by the Annals of Statistics. Larry Wasserman’s critique “A World Without Referees” merits a careful reply. I note it down for a future column, and continue with my program that could be called “A World of Happy Peers”.

I am convinced that most, if not all, ideas come in response to an adequate amount of frustration. In 2010 my refereeing-related frustration level was still sub-critical and so I missed on that IMS Bulletin’s call. I remember well the sequence of events, a couple of years after, that led to the needed super-critical state. It is part of our guild-type behavior not to share the details on any particular task. In fact, you already know the details—the counterproductive, time- and energy-consuming situation I got into is recurring, and you have almost surely been its victim on several occasions.

That nth iteration (with little variation) was finally sufficient for me. I woke up one morning with an idea that seemed good and practical (relatively easy to implement). I had some verbal encouragement from colleagues, and a lot of help from a family member. And so I set on a path that led… nowhere. I take that back: the path led to a number of things, including this series of columns. Yet the particularly annoying feature of peer reviewing stayed.

Here’s my point in “raw state” [inspired by sci fi] (with elaborations are postponed to the sequel): The advent of manuscript-central has greatly accelerated the transition of tasks (transfer of drafts/preprints, sending of prompting messages, transfer of reports, opinion letters, and copyright). From the very beginning, the same machine had the capacity to greatly enhance the most important (communication-wise and learning-wise) practical aspect of the review process, yet it completely ignored it. The imbalance thus created contributed to a notable increase in the frequency of careless or otherwise dishonest (peer-reviewed publications and) peer reports, gradually leading to questioning of the very foundations of the peer review. The speed at which the machine arrived and took over, combined with the ever-rising pressure which it exerts on the grassroots human power (a.k.a. peer reviewer), has captivated the rulers (a.k.a. publishing companies, editorial boards, hiring/promotion/award committees etc.) and has been distracting everyone ever since.

Still, there is hope on the horizon…