Real-world reproducibility study characterizing patients newly diagnosed with multiple myeloma using Clinical Practice Research Datalink, a UK-based electronic health records database

Abstract

Purpose: We evaluated the reproducibility of a study characterizing newly-diagnosed multiple myeloma (MM) patients within an electronic health records (EHR) database using different analytic tools.

Methods: We reproduced the findings of a descriptive cohort study using an iterative two-phase approach. In Phase I, a common protocol and statistical analysis plan (SAP) were implemented by independent investigators using the Aetion Evidence Platform® (AEP), a rapid-cycle analytics tool, and SAS statistical software as a gold standard for statistical analyses. Using the UK Clinical Practice Research Datalink (CPRD) dataset, the study included patients newly diagnosed with MM within primary care setting and assessed baseline demographics, conditions, drug exposure, and laboratory procedures. Phase II incorporated analysis revisions based on our initial comparison of the Phase I findings. Reproducibility of findings was evaluate by calculating the match rate and absolute difference in prevalence between the SAS and AEP study results.

Results: Phase I yielded slightly discrepant results, prompting amendments to SAP to add more clarity to operational decisions. After detailed specification of data and operational choices, exact concordance was achieved for the number of eligible patients (N = 2646), demographics, comorbidities (i.e., osteopenia, osteoporosis, cardiovascular disease [CVD], and hypertension), bone pain, skeletal-related events, drug exposure, and laboratory investigations in the Phase II analyses.

Conclusions: In this reproducibility study, a rapid-cycle analytics tool and traditional statistical software achieved near-exact findings after detailed specification of data and operational choices. Transparency and communication of the study design, operational and analytical choices between independent investigators were critical to achieve this reproducibility.

Keywords: epidemiologic methods; healthcare database; observational study; replication; reproducibility; transparency.