Pragmatic trial design elements showed a different impact on trial interpretation and feasibility than explanatory elements

Abstract

Objectives: To illustrate how pragmatic trial design elements or inserting explanatory trial elements in pragmatic trials affect validity, generalizability, precision, and operational feasibility.

Study design and setting: From illustrative examples identified through the IMI Get Real Consortium, we selected randomized drug trials with a pragmatic design feature. We searched all publications on these trials for information on how pragmatic trial design features affect validity, generalizability, precision, or feasibility.

Results: We present examples from the Salford lung study, International Suicide Prevention Trial, Sequenced Treatment Alternatives to Relieve Depression, and Cluster Randomized Usual care vs. Caduet Investigation Assessing Long-term-risk trial. These examples show that incorporating pragmatic trial design elements in trials may affect generalizability, precision and validity and may lead to operational challenges different from traditional explanatory trials. Inserting explanatory trial elements into pragmatic trials may also affect validity, generalizability, and operational feasibility, especially when these trial elements are incorporated in one arm of the trial only. Design choices that positively affect one of these domains (e.g., generalizability) may negatively affect others (e.g., feasibility).

Conclusion: Consequences of incorporating pragmatic or explanatory trial design elements in pragmatic trials should be explicitly considered and balanced for all relevant domains, including validity, generalizability, precision, and operational feasibility. Tools are needed to make these consequences more transparent.

Keywords: Explanatory trial; Generalizability; Operational feasibility; Pragmatic clinical trials; Precision; Real-world evidence; Trial design; Validity.