Vlada Limic closes her mini-series on ways to improve the peer review process. (Since she is taking over as Editor of the IMS Bulletin, Vlada is also ending her time as a contributing editor.) She writes:
Let me start by expanding on the concluding thought from Episode II [in the August 2016 issue: see https://imstat.org/2016/07/vladas-point-peer-review-ii-an-idea/].
You might already be aware of the fact that tens of thousands of our peers working in Astronomy and Space Sciences, Bioengineering and Biotechnology, Built Environment, Cardiovascular Medicine, or any of the other 34 research disciplines1 have access to a version of the web booth tool, integrated in the Frontiers platform, as described at http://home.frontiersin.org/about/publishing-model.
A different group of open-access journals, this time exclusively in physics, constitutes the SciPost publishing platform. Their entire editorial process exchanges seem to be publicly available2.While this platform certainly has some of the features evoked last time, it is also significantly different: discussions consist of (only) few longer exchanges, which strongly resemble peer reviewer reports or response letters, typically sent to the AEs in charge during the usual scientific review.
Finally, to our peers working in Computer Science and Applied Mathematics in Africa, Discrete Mathematics and Theoretical Computer Science, Number Theory, Human-Machine Interaction, Data Mining and Digital Humanities, Interdisciplinary Methodologies and Issues in Science, or in Algebraic Geometry, it may be useful to know that a web booth tool is integrated in the EpiSciences [https://www.episciences.org/] platform. Indeed, their figure—provided the term “Copyreaders” gets replaced with “Peer reviewers”—and the accompanying explanations suggest an access to a communication tool that functions quite similarly to the web booth (as recalled below). In particular, the peer reviewers and AEs in charge seem to stay anonymous, there seems to be no public trace of the direct exchanges between authors and peer-reviewers, and (apart from a web booth tool, and an “overlay” publishing from HAL or arXiv) their editorial process seems to be standard.
I anticipate you asking if there are any other examples. There are at least two possible (correct) answers, the first of which I give immediately. I do not know of any other examples, but this does not mean that the above list is exhaustive.
For those who continue reading, I have a rhetorical question. Did you know that Timothy Gowers (Cambridge University), Richard Taylor (IAS Princeton), Ravi Vakil (Stanford University), and Wendelin Werner (ETH Zürich) each expressed an interest in having a web booth tool available to peer reviewers in their area? My second and longer answer will likely make more sense in view of this information.
While a considerable need for spontaneous and direct anonymous peer reviewer + author(s) communication seems to exist in various areas of mathematics, at present there are not all that many web booths available to mathematicians or statisticians. Our community could wait for the tool to become integrated in most of our manuscript-centrals or publishing platforms, and this might happen eventually.
As already discussed, my idea is about an alternative path. More precisely, a web booth tool could exist freely and independently, and could be used (at the option of each reviewer) as a complement to the standard editorial process, regardless of the journal or the platform handling the paper. The image below illustrates this.
In fact, I have some good news. As I write this, a prototype tool is being made. By the time you read this, it will hopefully already be available online for anyone interested to play with, comment on, and constructively criticize it. Let me recall its main features here: the peer reviewer of any paper for any journal can create a discussion; the author (team) is invited to participate, and the journal contact is invited as an observer (if needed they can confirm that the invitation was issued by a designated referee for that paper); each discussion is private (can only be accessed by the above mentioned users); the names and contact information of the non-author participants are concealed from the authors; posts are grouped into threads (that can be started either by the referee or by one of the authors); files in various common formats can be attached to posts; each thread separately or the whole discussion can be printed out by any participant; at some point the discussion becomes obsolete and eventually deleted…
There is a more personal part of this story, mentioned implicitly in Episode I, that should be clarified. Four years ago it was my sister Mirna, a computer scientist, who single-handedly made a toy web booth. That early version had most of the above listed functionalities, but it was never publicly available, and was seen by very few people, most of whom are mentioned in this article.
The prototype is a creation of WebToolWorks, a company that Mirna founded two years ago. The material resources currently available for this project are limited. Some time ago, I reserved an inexpensive domain name www.refereeinitiative.work for the tool. In order to avoid web hosting costs, there will however be (at least initially) an automatic redirection from refereeinitiative.work to webtoolworks.com.
Our prototype is, and will remain, free for everyone to use. State-of-the-art security measures are built into the platform, and a security audit will be made as soon as sufficient funds become available. The prototype should greatly benefit from community feedback, so please do not hesitate to send us comments on its functionalities, name, user-friendliness, anything that should be changed or improved. We would be happy to accommodate your requests, as long as they are pertinent to the project, not time-consuming to implement, and do not compromise the security of the platform. It remains to be seen if the prototype will turn into a full-scale web booth at the service of every interested peer. In the meantime, much can be learned from this experiment, which may bring us closer to a world of happy peers (see Episode I).
My time as contributing editor is up. I am happy to respond to comments or questions either online [leave a comment below] or by email. While it will be difficult for the IMS Bulletin to give updates on this particular initiative in the next three years, I am counting on other channels.
Finally, I wish to express special thanks to Diana Gillooly at Cambridge University Press and to Ivar Martin of Knowen.org. Our exchanges during the last few months were very important for the writing of this column.
—
Footnotes:
1 listed at http://home.frontiersin.org/about/research-topics.
2 see, for example, submissions https://scipost.org/submission/1607.01285v1/ or https://scipost.org/submission/1606.09516v1/
Comments on “Vlada’s Point: Peer Review III—Web booth at your service”