In Cochrane reviews, risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance

Abstract

Objectives: The aim of the article was to analyze whether the risk of bias (RoB) judgments for allocation concealment in Cochrane systematic reviews (CSRs) were in line with recommendations from the Cochrane Handbook.

Study design and setting: From CSR, we extracted data about judgments and supporting comments about allocation concealment for each included randomized controlled trial (RCT). We compared whether judgments for supporting comments were in line with Cochrane Handbook recommendations.

Results: We analyzed judgments and comments of 721 CSRs in which 10,280 RCTs were included. By following the Cochrane Handbook guidance, we found that judgments for allocation concealment were discrepant for 2,928 trials (29%). Most discrepancies were made for trials where RoB was judged as low (2,693 trials; 92%). Cochrane authors used 66 categories of comments describing envelopes as a method of allocation concealment. In 66 envelope-related categories, describing RoB assessments from 1,529 (15%) of the analyzed RCTs, most of the judgments were low RoB, although only one of those categories is associated with low RoB according to the Cochrane Handbook. Twenty categories of supporting comments, including 642 trials, were related to randomization, not allocation concealment.

Conclusion: One-third of RoB judgments about allocation concealment in Cochrane reviews were discrepant from the Cochrane Handbook recommendations.

Keywords: Allocation concealment; Bias; Cochrane; Quality; Risk of bias; Standards; Systematic reviews.