An international modified Delphi process supported updating the web-based "right review" tool

Barbara Clyne; Melissa K Sharp; Michelle O' Neill; Danielle Pollock; Rosarie Lynch; Krystle Amog; Mairin Ryan; Susan M Smith; Kamal Mahtani; Andrew Booth; Christina Godfrey; Zachary Munn; Andrea C Tricco

doi:10.1016/j.jclinepi.2024.111333

An international modified Delphi process supported updating the web-based "right review" tool

J Clin Epidemiol. 2024 Mar 24:170:111333. doi: 10.1016/j.jclinepi.2024.111333. Online ahead of print.

Authors

Affiliations

¹ Department of Public Health and Epidemiology, RCSI University of Medicine and Health Sciences, Dublin, Ireland. Electronic address: barbaraclyne@rcsi.com.
² Department of Public Health and Epidemiology, RCSI University of Medicine and Health Sciences, Dublin, Ireland.
³ Health Information and Quality Authority (HIQA), Dublin, Ireland.
⁴ Health Evidence Synthesis, Recommendations and Impact, School of Public Health, The University of Adelaide Faculty of Health and Medical Sciences, Adelaide, Australia.
⁵ Department of Health, Dublin, Ireland.
⁶ Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada; Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada.
⁷ Department of Public Health and Primary Care, Trinity College Dublin, Dublin, Ireland.
⁸ Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK.
⁹ Sheffield Centre for Health and Related Research (SCHARR), University of Sheffield, Sheffield, UK; University of Limerick, Limerick, Ireland.
¹⁰ Queen's Collaboration for Health Care Quality: A JBI Centre of Excellence, School of Nursing, Queen's University, Kingston, Ontario, Canada.
¹¹ Queen's Collaboration for Health Care Quality: A JBI Centre of Excellence, School of Nursing, Queen's University, Kingston, Ontario, Canada; Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada; Epidemiology Division and Institute for Health, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada.

PMID: 38522755
DOI: 10.1016/j.jclinepi.2024.111333

Abstract

Objectives: The proliferation of evidence synthesis methods makes it challenging for reviewers to select the ''right'' method. This study aimed to update the Right Review tool (a web-based decision support tool that guides users through a series of questions for recommending evidence synthesis methods) and establish a common set of questions for the synthesis of both quantitative and qualitative studies (https://rightreview.knowledgetranslation.net/).

Study design and setting: A 2-round modified international electronic modified Delphi was conducted (2022) with researchers, health-care providers, patients, and policy makers. Panel members rated the importance/clarity of the Right Review tool's guiding questions, evidence synthesis type definitions and tool output. High agreement was defined as at least 70% agreement. Any items not reaching high agreement after round 2 were discussed by the international Project Steering Group.

Results: Twenty-four experts from 9 countries completed round 1, with 12 completing round 2. Of the 46 items presented in round 1, 21 reached high agreement. Twenty-seven items were presented in round 2, with 8 reaching high agreement. The Project Steering Group discussed items not reaching high agreement, including 8 guiding questions, 9 review definitions (predominantly related to qualitative synthesis), and 2 output items. Three items were removed entirely and the remaining 16 revised and edited and/or combined with existing items. The final tool comprises 42 items; 9 guiding questions, 25 evidence synthesis definitions and approaches, and 8 tool outputs.

Conclusion: The freely accessible Right Review tool supports choosing an appropriate review method. The design and clarity of this tool was enhanced by harnessing the Delphi technique to shape ongoing development. The updated tool is expected to be available in Quarter 1, 2025.

Keywords: Delphi; Evidence synthesis; Qualitative evidence synthesis; Research methods; Systematic reviews; Web-based decision tool.