Discussing the future of open peer-review : a survey of journals in the JCR Public , Environmental and Occupational Health category

Introduction Open Peer-Review (OPR) is an “umbrella” term for a number of innovations in the peer-review process that aim to make traditional peer-review more transparent, inclusive, accountable and reliable. Although for almost 30 years there has been a lot of discussion in the academic community about the need of some form of openness in the peer-review procedure (1), OPR has started to expand more recently, generally encouraged by the Open Science movement that promotes increasing transparency and participation in scientific practices (2). OPR aims to open up a system traditionally closed in many ways: the identities of both author and reviewer can be revealed to each other; the manuscripts can be made immediately available online, before any official peer-review procedure; the reviewer reports can be published together with the articles; experts and the general public can be allowed to comment and contribute to the review process; the reviewers may be algorithmically determined and requests sent out automatically and so on (3). Many journals already run some form of OPR, though this is implemented differently and in different stages of the peer-review process. OPR can be preor postpublication, in fact, both pre-prints and published papers can be commented on; it can be “de-coupled”, when reviews are performed by different peer-review services, such as Peerage of Science, on request by authors before submission (4) and it can be “portable”, when peer-review comments about papers are passed from one journal to another, in journals of the same publisher (5). The debate on OPR application is growing faster, boosted by new technologies, open access models and new circumstances, like the COVID-19 pandemic. Today the reception of OPR is changing as the concepts of open interaction, open discussion and participation are perceived positively by a new generation of authors, editors and reviewers (6). And yet the attitudes in its regard can vary widely: despite the need for a change, the paper findings suggest that the debate on the future of peer-review is still ongoing. In this article the authors, who are members of the Editorial team of the Annali dell’Istituto Superiore di Sanità (the official Journal of the Italian National Institute of Health, Rome, Italy), analyse the different forms of OPR practiced by scholarly journals included in the JCR Public, Environmental and Occupational Health category. This is followed by a discussion of the implications of our findings for the future of OPR. Abstract The journals listed in the JCR Public, Environmental and Occupational Health category are examined by the authors in order to check how many of them practice some kind of Open Peer-Review (OPR). An overview of the different OPR methods identified is given: a variety of practices considered as OPR even though the number of journals using them is very small. Furthermore, the possible future evolution of OPR is examined.


Introduction
Open Peer-Review (OPR) is an "umbrella" term for a number of innovations in the peer-review process that aim to make traditional peer-review more transparent, inclusive, accountable and reliable. Although for almost 30 years there has been a lot of discussion in the academic community about the need of some form of openness in the peer-review procedure (1), OPR has started to expand more recently, generally encouraged by the Open Science movement that promotes increasing transparency and participation in scientific practices (2). OPR aims to open up a system traditionally closed in many ways: the identities of both author and reviewer can be revealed to each other; the manuscripts can be made immediately available online, before any official peer-review procedure; the reviewer reports can be published together with the articles; experts and the general public can be allowed to comment and contribute to the review process; the reviewers may be algorithmically determined and requests sent out automatically and so on (3). Many journals already run some form of OPR, though this is implemented differently and in different stages of the peer-review process. OPR can be pre-or post-publication, in fact, both pre-prints and published papers can be commented on; it can be "de-coupled", when reviews are performed by different peer-review services, such as Peerage of Science, on request by authors before submission (4) and it can be "portable", when peer-review comments about papers are passed from one journal to another, in journals of the same publisher (5). The debate on OPR application is growing faster, boosted by new technologies, open access models and new circumstances, like the COVID-19 pandemic. Today the reception of OPR is changing as the concepts of open interaction, open discussion and participation are perceived positively by a new generation of authors, editors and reviewers (6). And yet the attitudes in its regard can vary widely: despite the need for a change, the paper findings suggest that the debate on the future of peer-review is still ongoing. In this article the authors, who are members of the Editorial team of the Annali dell'Istituto Superiore di Sanità (the official Journal of the Italian National Institute of Health, Rome, Italy), analyse the different forms of OPR practiced by scholarly journals included in the JCR Public, Environmental and Occupational Health category. This is followed by a discussion of the implications of our findings for the future of OPR.

Methods
The study examines the 193 journals listed in the Public, Environmental and Occupational Health category in the 2019 edition of the Journal Citation Reports (JCR, published by Clarivate Analytics) in order to check how many of them practice some kind of OPR and of what type. This category was chosen because it is the category in which the journal Annali dell'Istituto Superiore di Sanità is listed. In order to obtain information on journals' peer-review processes, the sections Information for authors and About of each journal website were checked. The analysis was carried out with the version of these pages as accessed on September 9, 2020. The authors would like to underline that journal policies and practices may have changed in the meantime. These sections were examined to find any statements related to peer-review, more precisely any with a specific open or non-traditional peer-review practice. Given the small number of cases involved a systematic categorization was not adopted.

Results
Most of the journals listed in the JCR Public, Environmental and Occupational Health category still use tra-ditional peer-review. As of September 9, 2020, it was found that, out of 193 journals in that category, only 9 journals (4.8%) practice some kind of OPR. None of the publishers of these journals have implemented OPR in the same manner. Table 1 offers a list of the journals that declare themselves to implement different OPR practices, and the exact terminology used for the description of the adopted OPR method is reported in the last column. The majority (six) of the journals listed in Table 1 are Open Access titles published by a born open access publisher (MDPI, BMC, Frontiers). However, the remaining three journals are not Open Access titles from a diverse group of publishers, suggesting that experiments with peer-review models are in large part independent of the Open Access model. Every publisher explains its OPR policy with a detailed statement, reflecting the diversity of practices and the consequent lack of a widely shared terminology. This is well illustrated by comparing the introductory statements of the various policies: (Frontiers Collaborative Peer-review). The most common type of OPR, defined simply as Open Peer-review, usually, but not always, means that both the reviewer and the author are known to each other during the peer-review process. Nevertheless, some publishers use this term to refer to the practice of publishing reviewers' names on the article page, or peerreview reports (signed or anonymous) alongside the article or on Publons, a specific platform born as a place to help researchers get recognition for their often-hidden peer-review contributions (7).

Discussion
OPR was first mentioned as a possible alternative method to classic peer-reviewing at the end of the 80s. Nowadays, about thirty years later, it has become an increasingly emerging practice in scholarly publishing. Many are the factors leading to these changes, among them: the general tendency towards a greater openness in all fields including science, the many faults and bias of traditional peer-review which has been under scrutiny in the last decades, and certainly the availability of new technologically advanced platforms for science journals. Even the International Committee of Medical Journal Editors (8) is timidly supporting some new forms of post-publication OPR. In the Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals (Updated December 2019) we read "Some people believe that true scientific peer-review begins only on the date a paper is published. In that spirit, medical journals should have a mechanism for readers to submit comments, questions, or criticisms about published articles, and authors have a responsibility to respond appropriately and cooperate with any requests from the journal for data or additional information should questions about the paper arise after publication". Different forms of OPR implementation co-exist at present (9), but OPR in any form is still adopted by a minority of scholarly journals, as shown by the results of this study which are in line with those by Kwee et al. (10) who investigated the peer-review practices of journals included in another category of the 2018 JCR Radiology, nuclear medicine and medical imaging, where only two (1.7%) out 119 journals used an OPR process. Further investigation is needed on the implementation of OPR procedures in other JCR categories, and in open access vs non open access journals, but the lack of clear definitions makes these investigations more difficult to conduct and to evaluate. The question arising spontaneously is why so few publications are experimenting with OPR. Several reasons might help explaining it: among them, the permanence of a cultural reluctance and concern by reviewers in exposing themselves and in being identified while possibly criticizing the work of more experienced and qualified authors (4). Another reason might be more practical: the setting up of a new peer-review system can be money and time consuming, and therefore it is, at the moment, attempted mainly by big publishers. Moreover, finding qualified and dedicated reviewers becomes harder and it is likely that the fear of an impact of policy changes on an editor's ability to recruit them could contribute to the low implementation of OPR policies (11).

Conclusion
In conclusion, it is foreseeable that OPR will continue to expand its reach in the coming years. However, in order for scholarly journals to fully adopt new forms of OPR, its pros and cons should be further addressed and studied, taking into consideration issues like: trust in the peer-review system, transparency vs accuracy of the comments, conflicts of interest, acknowledgment of reviewers, reviewer's motivation and feelings. To ensure a sound OPR, a consensus should be reached on: • a clear and accurate definition of the different OPR methods; • guidelines agreed at international level; • standardised assessment criteria; • ethical issues and best practice recommendations. In the meantime, whichever OPR practice is approved by the Editors, this should be clearly reported in the peer-review process description page of each journal, as recommended by the ICMJE. The dissemination of scientific knowledge should be accelerated, as the COVID-19 pandemic has clearly shown, but trust and fairness need to be guaranteed in the whole publication process. Perhaps, in the future of scholarly publications, open peer-review will find the way to become a better peer-review.