A causal roadmap for generating high-quality real-world evidence

Lauren E Dang; Susan Gruber; Hana Lee; Issa J Dahabreh; Elizabeth A Stuart; Brian D Williamson; Richard Wyss; Iván Díaz; Debashis Ghosh; Emre Kıcıman; Demissie Alemayehu; Katherine L Hoffman; Carla Y Vossen; Raymond A Huml; Henrik Ravn; Kajsa Kvist; Richard Pratley; Mei-Chiung Shih; Gene Pennello; David Martin; Salina P Waddy; Charles E Barr; Mouna Akacha; John B Buse; Mark van der Laan; Maya Petersen

doi:10.1017/cts.2023.635

A causal roadmap for generating high-quality real-world evidence

J Clin Transl Sci. 2023 Sep 22;7(1):e212. doi: 10.1017/cts.2023.635. eCollection 2023.

Authors

Lauren E Dang¹, Susan Gruber², Hana Lee³, Issa J Dahabreh⁴, Elizabeth A Stuart⁵, Brian D Williamson⁶, Richard Wyss⁷, Iván Díaz⁸, Debashis Ghosh⁹, Emre Kıcıman¹⁰, Demissie Alemayehu¹¹, Katherine L Hoffman¹², Carla Y Vossen¹³, Raymond A Huml¹⁴, Henrik Ravn¹⁵, Kajsa Kvist¹⁵, Richard Pratley¹⁶, Mei-Chiung Shih^{17

18}, Gene Pennello¹⁹, David Martin²⁰, Salina P Waddy²¹, Charles E Barr^{22

23}, Mouna Akacha²⁴, John B Buse²⁵, Mark van der Laan¹, Maya Petersen¹

Affiliations

¹ Department of Biostatistics, University of California, Berkeley, CA, USA.
² TL Revolution, Cambridge, MA, USA.
³ Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, MD, USA.
⁴ CAUSALab, Department of Epidemiology and Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
⁵ Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
⁶ Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.
⁷ Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
⁸ Division of Biostatistics, Department of Population Health, New York University Grossman School of Medicine, New York, NY, USA.
⁹ Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
¹⁰ Microsoft Research, Redmond, WA, USA.
¹¹ Global Biometrics and Data Management, Pfizer Inc., New York, NY, USA.
¹² Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA.
¹³ Syneos Health Clinical Solutions, Amsterdam, The Netherlands.
¹⁴ Syneos Health Clinical Solutions, Morrisville, NC, USA.
¹⁵ Novo Nordisk, Søborg, Denmark.
¹⁶ AdventHealth Translational Research Institute, Orlando, FL, USA.
¹⁷ Cooperative Studies Program Coordinating Center, VA Palo Alto Health Care System, Palo Alto, CA, USA.
¹⁸ Department of Biomedical Data Science, Stanford University, Stanford, CA, USA.
¹⁹ Division of Imaging Diagnostics and Software Reliability, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA.
²⁰ Global Real World Evidence Group, Moderna, Cambridge, MA, USA.
²¹ National Center for Advancing Translational Sciences, Bethesda, MD, USA.
²² Graticule Inc., Newton, MA, USA.
²³ Adaptic Health Inc., Palo Alto, CA, USA.
²⁴ Novartis Pharma AG, Basel, Switzerland.
²⁵ Division of Endocrinology, Department of Medicine, University of North Carolina, Chapel Hill, NC, USA.

Abstract

Increasing emphasis on the use of real-world evidence (RWE) to support clinical policy and regulatory decision-making has led to a proliferation of guidance, advice, and frameworks from regulatory agencies, academia, professional societies, and industry. A broad spectrum of studies use real-world data (RWD) to produce RWE, ranging from randomized trials with outcomes assessed using RWD to fully observational studies. Yet, many proposals for generating RWE lack sufficient detail, and many analyses of RWD suffer from implausible assumptions, other methodological flaws, or inappropriate interpretations. The Causal Roadmap is an explicit, itemized, iterative process that guides investigators to prespecify study design and analysis plans; it addresses a wide range of guidance within a single framework. By supporting the transparent evaluation of causal assumptions and facilitating objective comparisons of design and analysis choices based on prespecified criteria, the Roadmap can help investigators to evaluate the quality of evidence that a given study is likely to produce, specify a study to generate high-quality RWE, and communicate effectively with regulatory agencies and other stakeholders. This paper aims to disseminate and extend the Causal Roadmap framework for use by clinical and translational researchers; three companion papers demonstrate applications of the Causal Roadmap for specific use cases.

Keywords: Causal inference; estimands; machine learning; real-world evidence; sensitivity analysis; simulations.

Publication types

Review