External factors may influence Cochrane reviewers when classifying the risk of bias of original reports

Lorenzo Bertizzolo; Patrick M Bossuyt; Ignacio Atal; Philippe Ravaud; Agnès Dechartres

doi:10.1016/j.jclinepi.2020.03.008

External factors may influence Cochrane reviewers when classifying the risk of bias of original reports

J Clin Epidemiol. 2020 Jul:123:1-8. doi: 10.1016/j.jclinepi.2020.03.008. Epub 2020 Mar 19.

Authors

Lorenzo Bertizzolo¹, Patrick M Bossuyt², Ignacio Atal³, Philippe Ravaud⁴, Agnès Dechartres⁵

Affiliations

¹ INSERM, U1153 Epidemiology and Biostatistics Sorbonne Paris Cité Research Center (CRESS), Methods of Therapeutic Evaluation of Chronic Diseases Team (METHODS), Paris F-75004, France; Paris Descartes University, Sorbonne Paris Cité, France; Department of Clinical Epidemiology and Biostatistics, Academic University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands. Electronic address: lorenzo.bertizzolo@gmail.com.
² Department of Clinical Epidemiology and Biostatistics, Academic University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands.
³ INSERM, U1153 Epidemiology and Biostatistics Sorbonne Paris Cité Research Center (CRESS), Methods of Therapeutic Evaluation of Chronic Diseases Team (METHODS), Paris F-75004, France; Paris Descartes University, Sorbonne Paris Cité, France; Cochrane France, Paris, France; Centre de Recherches Interdisciplinaires (CRI), Université Paris Descartes, University of Paris, Paris, France.
⁴ INSERM, U1153 Epidemiology and Biostatistics Sorbonne Paris Cité Research Center (CRESS), Methods of Therapeutic Evaluation of Chronic Diseases Team (METHODS), Paris F-75004, France; Paris Descartes University, Sorbonne Paris Cité, France; Cochrane France, Paris, France; Centre d'Épidémiologie Clinique, Hôpital Hôtel Dieu AP-HP (Assistance Publique - Hôpitaux de Paris), Paris, France; Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France; Department of Epidemiology, Columbia University, Mailman School of Public Health, New York, USA.
⁵ Sorbonne Université, Institut National de la Santé et de la Recherche Médicale (INSERM), Institut Pierre Louis d'Epidémiologie et de Santé Publique, AP-HP. Sorbonne Université, Hôpital Pitié Salpêtrière, Département de Santé Publique, Centre de Pharmacoépidémiologie de l'AP-HP (Cephepi), CIC-1422, F75013, Paris, France.

PMID: 32200041
DOI: 10.1016/j.jclinepi.2020.03.008

Abstract

Objective: The objective of the study was to explore contextual factors associated with high or low risk-of-bias judgment in case of incomplete or unclear information in study reports.

Study design and setting: Research-on-research study, using matched case-control design, with a sample of 304 randomized controlled trials (RCTs) included in two Cochrane reviews for which there was disagreement on the risk-of-bias judgment related to incomplete or unclear information in the study report. A case was defined as an RCT judged at high or low risk of bias; a control was the same RCT judged at unclear risk. We used a conditional logistic regression model for analysis.

Results: Review authors being also authors of the RCT were more likely to assess an item at low risk of bias than unclear (OR: 11.71; 95% CI: 1.39-98.76). Earlier trials in a review were more often assigned a low risk (OR: 0.37; [0.15-0.96]). Review groups and authors that had completed a lower number of reviews slightly more often assigned a low risk, whereas others reported "unclear" (OR: 0.97, [95% CI: 0.95-0.99] for groups) and 0.97 (95% CI: 0.95-0.998) for authors).

Conclusions: Risk-of-bias assessment of RCTs in case of incomplete or unclear information may be affected by contextual factors.

Keywords: Matched case-control; Reporting; Research on research; Risk of bias; Systematic reviews.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Authorship*
Bias
Case-Control Studies
Humans
Judgment
Randomized Controlled Trials as Topic / statistics & numerical data*
Systematic Reviews as Topic*