The 2-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

Catalin Tufanaru; Didi Surian; Anna Mae Scott; Paul Glasziou; Enrico Coiera

doi:10.1016/j.jclinepi.2023.10.013

The 2-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

J Clin Epidemiol. 2024 Jan:165:111197. doi: 10.1016/j.jclinepi.2023.10.013. Epub 2023 Oct 23.

Authors

Catalin Tufanaru¹, Didi Surian², Anna Mae Scott³, Paul Glasziou³, Enrico Coiera¹

Affiliations

¹ Australian Institute of Health Innovation, Level 6, Macquarie University, 75 Talavera Road, North Ryde, New South Wales 2109, Australia.
² Australian Institute of Health Innovation, Level 6, Macquarie University, 75 Talavera Road, North Ryde, New South Wales 2109, Australia. Electronic address: didi.surian@mq.edu.au.
³ Institute for Evidence-Based Healthcare, Bond University, 14 University Drive, Robina, Queensland 4226, Australia.

PMID: 37879542
DOI: 10.1016/j.jclinepi.2023.10.013

Abstract

Objective: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method.

Methods: A Preferred Reporting Items for Systematic reviews and Meta-Analyses compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores (FSs) were calculated for review methods and outcomes. Barriers to completing the replication were identified.

Results: The review was completed over 63 person-hours (11 workdays/15 calendar days). A FS of 0.95 was achieved for the methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the outcomes, with 6 (of 7) only partially replicated and one task was not replicated. Nonreplication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study.

Conclusion: A 2weekSR study was replicated by a small team of three reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.

Keywords: 2-Week systematic review; Automation tool; Barriers; Facilitators; Replication fidelity; Replication study.