Marginal structural models and other analyses allow multiple estimates of treatment effects in randomized clinical trials: Meta-epidemiological analysis

Hannah Ewald; Benjamin Speich; Aviv Ladanie; Heiner C Bucher; John P A Ioannidis; Lars G Hemkens

doi:10.1016/j.jclinepi.2018.11.001

Marginal structural models and other analyses allow multiple estimates of treatment effects in randomized clinical trials: Meta-epidemiological analysis

J Clin Epidemiol. 2019 Mar:107:12-26. doi: 10.1016/j.jclinepi.2018.11.001. Epub 2018 Nov 10.

Authors

Hannah Ewald¹, Benjamin Speich², Aviv Ladanie³, Heiner C Bucher², John P A Ioannidis⁴, Lars G Hemkens⁵

Affiliations

¹ Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel 4031, Switzerland; Swiss Tropical and Public Health Institute, Basel 4051, Switzerland; University Medical Library, University of Basel, Basel 4051, Switzerland.
² Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel 4031, Switzerland.
³ Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel 4031, Switzerland; Swiss Tropical and Public Health Institute, Basel 4051, Switzerland.
⁴ Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Palo Alto, CA 94305, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA 94305, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA 94305, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA 94305, USA.
⁵ Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel 4031, Switzerland. Electronic address: Lars.Hemkens@usb.ch.

PMID: 30423375
DOI: 10.1016/j.jclinepi.2018.11.001

Abstract

Objectives: To determine how marginal structural models (MSMs), which are increasingly used to estimate causal effects, are used in randomized clinical trials (RCTs) and compare their results with those from intention-to-treat (ITT) or other analyses.

Study design and setting: We searched PubMed, Scopus, citations of key references, and Clinicaltrials.gov. Eligible RCTs reported clinical effects based on MSMs and at least one other analysis.

Results: We included 12 RCTs reporting 138 analyses for 24 clinical questions. In 19/24 (79%), MSM-based and other effect estimates were all in the same direction, 22/22 had overlapping 95% confidence intervals (CIs), and in 19/22 (86%), the MSM effect estimate lay within all 95% CIs of all other effects (in two cases no CIs were reported). For the same clinical question, the largest effect estimate from any analysis was 1.19-fold (median; interquartile range 1.13-1.34) larger than the smallest. All MSM and ITT effect estimates were in the same direction and had overlapping 95% CIs. In 71% (12/17), they also agreed on the presence of statistical significance. MSM-based effect estimates deviated more from the null than those based on ITT (P = 0.18). The effect estimates of both approaches differed 1.12-fold (median; interquartile range 1.02-1.22).

Conclusions: MSMs provided largely similar effect estimates as other available analyses. Nevertheless, some of the differences in effect estimates or statistical significance may become important in clinical decision-making, and the multiple estimates require utmost attention of possible selective reporting bias.

Keywords: Intention-to-treat; Marginal structural models; Prespecification; Randomized controlled trial; Reporting; Selective reporting; Vibration of effects.

Publication types

Meta-Analysis
Research Support, Non-U.S. Gov't

MeSH terms

Data Interpretation, Statistical
Epidemiologic Methods*
Humans
Intention to Treat Analysis
Models, Theoretical
Randomized Controlled Trials as Topic / methods*
Research Design
Treatment Outcome