Systematic Review of the "Pragmatism" of Pragmatic Critical Care Trials

Abstract

To assess the pragmatism of published critical care randomized controlled trials self-described as pragmatic using a validated tool.

Data sources: Medical Literature Analysis and Retrieval Online database and PubMed interface from inception to November 1, 2021.

Study selection: We performed a systematic search of randomized controlled trials evaluating interventions for critically ill adults that self-identified as pragmatic in title or abstract.

Data extraction: Reviewers independently performed study selection and data extraction in duplicate; discrepancies were resolved by consensus. Pragmatism was assessed independently in duplicate by trained reviewers using the Pragmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2), a validated tool designed to represent how explanatory/pragmatic a trial is on the pragmatic to explanatory continuum. Trials were scored in nine domains on a 5-point continuum (from 1 = very explanatory to 5 = very pragmatic). Discrepancies of greater than 2 points were adjudicated by consensus discussion.

Data synthesis: The search resulted in 284 studies; 56 met eligibility criteria. Forty-one of the trials had a discrepancy in at least one domain that required consensus discussion, most commonly in domains of eligibility and follow-up. Twelve studies (21.4%) were scored as "overall pragmatic," defined as score of greater than 4 in five domains provided the scores in the remaining domains were three. The overall PRECIS-2 score of self-identified pragmatic studies increased from 1995 to 2021 suggesting increasing pragmatism over time. Pragmatic trials were more likely to have a waiver of informed consent (p = 0.05).

Conclusions: The number and pragmatism of self-identified pragmatic trials have increased, particularly in the past decade. However, less than one-quarter of these trials that use the term pragmatic in title or abstract were retrospectively rated as pragmatic. Our results support the concept that trials are designed on a spectrum of pragmatic to explanatory. Advances in the design and reporting of critical care trials are needed to ensure their real-world applicability.

Keywords: critical illness; pragmatic clinical trial; randomized controlled trial.