Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives

Enrique Orduña-Malea; Adolfo Alonso-Arroyo; José-Antonio Ontalba-Ruipérez; Ferrán Catalá-López

doi:10.1007/s11192-022-04542-z

Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives

Scientometrics. 2023;128(1):407-440. doi: 10.1007/s11192-022-04542-z. Epub 2022 Oct 17.

Authors

Enrique Orduña-Malea¹, Adolfo Alonso-Arroyo^{2

3}, José-Antonio Ontalba-Ruipérez¹, Ferrán Catalá-López^{4

5

6}

Affiliations

¹ Department of Audiovisual Communication, Documentation and History of Art, Universitat Politècnica de València, Valencia, Spain.
² Department of History of Science and Documentation, University of Valencia, Valencia, Spain.
³ Joint Research Unit CSIC-University of Valencia, UISYS, Valencia, Spain.
⁴ Department of Health Planning and Economics, National School of Public Health, Institute of Health Carlos III, Madrid, Spain.
⁵ Department of Medicine, University of Valencia/INCLIVA Health Research Institute and CIBERSAM, Valencia, Spain.
⁶ Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada.

Abstract

Reporting guidelines are tools to help improve the transparency, completeness, and clarity of published articles in health research. Specifically, the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statements provide evidence-based guidance on what to include in randomised trial articles and protocols to guarantee the efficacy of interventions. These guidelines are subsequently described and discussed in journal articles and used to produce checklists. Determining the online impact (i.e., number and type of links received) of these articles can provide insights into the dissemination of reporting guidelines in broader environments (web-at-large) than simply that of the scientific publications that cite them. To address the technical limitations of link analysis, here the Debug-Validate-Access-Find (DVAF) method is designed and implemented to measure different facets of the guidelines' online impact. A total of 65 articles related to 38 reporting guidelines are taken as a baseline, providing 240,128 URL citations, which are then refined, analysed, and categorised using the DVAF method. A total of 15,582 links to journal articles related to the CONSORT and SPIRIT initiatives were identified. CONSORT 2010 and SPIRIT 2013 were the reporting guidelines that received most links (URL citations) from other online objects (5328 and 2190, respectively). Overall, the online impact obtained is scattered (URL citations are received by different article URL IDs, mainly from link-based DOIs), narrow (limited number of linking domain names, half of articles are linked from fewer than 29 domain names), concentrated (links come from just a few academic publishers, around 60% from publishers), non-reputed (84% of links come from dubious websites and fake domain names) and highly decayed (89% of linking domain names were not accessible at the time of the analysis). In light of these results, it is concluded that the online impact of these guidelines could be improved, and a set of recommendations are proposed to this end.

Supplementary information: The online version contains supplementary material available at 10.1007/s11192-022-04542-z.

Keywords: Altmetrics; Article-level metrics; CONSORT; Clinical trials; Link analysis; Online impact; Reporting guidelines; SPIRIT; Scientific impact; Webometrics.