A structured, journal-led peer-review mentoring program enhances peer review training

Ariel Maia Lyons-Warren; Whitley W Aamodt; Kathleen M Pieper; Roy E Strowd

doi:10.1186/s41073-024-00143-x

A structured, journal-led peer-review mentoring program enhances peer review training

Res Integr Peer Rev. 2024 Mar 8;9(1):3. doi: 10.1186/s41073-024-00143-x.

Authors

Ariel Maia Lyons-Warren¹, Whitley W Aamodt², Kathleen M Pieper³, Roy E Strowd⁴

Affiliations

¹ Department of Pediatrics, Section of Neurology; Clinical Care Center, Baylor College of Medicine, Houston, USA. lyonswar@bcm.edu.
² Department of Neurology, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA.
³ American Academy of Neurology, Minneapolis, USA.
⁴ Departments of Neurology and Internal Medicine, Wake Forest University School of Medicine, Winston-Salem, USA.

Abstract

Background: Peer review is essential to the advancement of knowledge. However, training on how to conduct peer review is limited, unorganized, and not well studied. Thus, we sought to determine if a structured mentored peer-review program improved peer review training as measured by multiple quantitative and qualitative assessments.

Methods: This pre-post intervention study enrolled 55 mentees across 5 cohorts from 2020 to 2023. Each cohort completed pre-program evaluations, participated in 2 mentored reviews, and completed post-program evaluations over 6 months. Mentors and mentees completed pre-program demographic and review experience questionnaires. Outcome measures included (1) total and sub-scores on the modified Review Quality Index (mRQI) applied to the same pre-selected research manuscript reviewed by mentees both pre and post intervention, (2) mentee self-perceived comfort with and understanding of the review process using a custom questionnaire, and (3) mentor satisfaction surveys. Pre- and post-program measures were compared using the Wilcoxon signed-rank test.

Results: Post-program total modified RQI score (median (IQR) = 31 (26.3-35.8)) was higher than pre-program total score (26.6 (19.7-29.7)) for the 42 mentees who completed both pre- and post-program reviews. Mentees reported improved perception of review (median (IQR) pre = 4 (3-4), post = 5 (4-5)) and editorial processes (pre = 3 (2-4), post = 4 (4-5)) as well as self-perceived confidence in completing an independent review of both scientific (median (IQR) pre = 2 (2-3), post = 4 (4-4)) and non-scientific (pre = 3 (2-4), post = 4 (4-5)) manuscripts following program participation. p < 0.0001 for all scores noted. Mentors reported high scores for enjoyment (median (range) 5/5 (3-5)) and interest in repeat participation (5/5 (2-5)).

Conclusions: A 6-month structured mentored-review program including 2 mentored reviews improves peer review training as measured by the modified RQI as well as participant self-perceived understanding of publication science with high mentor satisfaction.

Keywords: Journalology; Mentorship; Peer review; Research; Surveys and Questionnaires; Training.