A prospective study on an innovative online forum for peer reviewing of surgical science

Martin Almquist; Regula S von Allmen; Dan Carradice; Steven J Oosterling; Kirsty McFarlane; Bas Wijnhoven

doi:10.1371/journal.pone.0179031

A prospective study on an innovative online forum for peer reviewing of surgical science

PLoS One. 2017 Jun 29;12(6):e0179031. doi: 10.1371/journal.pone.0179031. eCollection 2017.

Authors

Martin Almquist¹, Regula S von Allmen², Dan Carradice³, Steven J Oosterling⁴, Kirsty McFarlane⁵, Bas Wijnhoven⁶

Affiliations

¹ Dept of Surgery, Skåne University Hospital, Lund, Sweden; Institution of Clinical Sciences, Lund University, Lund, Sweden.
² Clinic for Vascular Surgery, Kantonsspital St Gallen, Switzerland.
³ Academic Vascular Surgical Unit, Hull York Medical School, Hull, United Kingdom.
⁴ Dept of Surgery, Spaarne Gasthuis Hospital, Haarlem, The Netherlands.
⁵ Associate Editor, BJS, John Wiley and Sons, Oxford, United Kingdom.
⁶ Department of Surgery, Erasmus MC-University Medical Center Rotterdam, The Netherlands.

Abstract

Background: Peer review is important to the scientific process. However, the present system has been criticised and accused of bias, lack of transparency, failure to detect significant breakthrough and error. At the British Journal of Surgery (BJS), after surveying authors' and reviewers' opinions on peer review, we piloted an open online forum with the aim of improving the peer review process.

Methods: In December 2014, a web-based survey assessing attitudes towards open online review was sent to reviewers with a BJS account in Scholar One. From April to June 2015, authors were invited to allow their manuscripts to undergo online peer review in addition to the standard peer review process. The quality of each review was evaluated by editors and editorial assistants using a validated instrument based on a Likert scale.

Results: The survey was sent to 6635 reviewers. In all, 1454 (21.9%) responded. Support for online peer review was strong, with only 10% stating that they would not subject their manuscripts to online peer review. The most prevalent concern was about intellectual property, being highlighted in 118 of 284 comments (41.5%). Out of 265 eligible manuscripts, 110 were included in the online peer review trial. Around 7000 potential reviewers were invited to review each manuscript. In all, 44 of 110 manuscripts (40%) received 100 reviews from 59 reviewers, alongside 115 conventional reviews. The quality of the open forum reviews was lower than for conventional reviews (2.13 (± 0.75) versus 2.84 (± 0.71), P<0.001).

Conclusion: Open online peer review is feasible in this setting, but it attracts few reviews, of lower quality than conventional peer reviews.

MeSH terms

Adult
Female
Humans
Male
Middle Aged
Online Systems*
Peer Review, Health Care*
Prospective Studies
Surgical Procedures, Operative*

Grants and funding

All authors had financial support from the BJS/John Wiley and Sons for the submitted work; there were no other financial relationships with any organisations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work. The funder provided support in the form of salary for one of the co-authors (KMcF), but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of the authors are articulated in the ‘author contributions’ section.