Controversy and Debate on Meta-epidemiology. Paper 1: Treatment effect sizes vary in randomized trials depending on the type of outcome measure

Dorthe B Berthelsen; Elisabeth Ginnerup-Nielsen; Carsten Juhl; Hans Lund; Marius Henriksen; Asbjørn Hróbjartsson; Sabrina M Nielsen; Marieke Voshaar; Robin Christensen

doi:10.1016/j.jclinepi.2019.10.016

Controversy and Debate on Meta-epidemiology. Paper 1: Treatment effect sizes vary in randomized trials depending on the type of outcome measure

J Clin Epidemiol. 2020 Jul:123:27-38. doi: 10.1016/j.jclinepi.2019.10.016. Epub 2020 Mar 23.

Authors

Dorthe B Berthelsen¹, Elisabeth Ginnerup-Nielsen¹, Carsten Juhl², Hans Lund³, Marius Henriksen⁴, Asbjørn Hróbjartsson⁵, Sabrina M Nielsen¹, Marieke Voshaar⁶, Robin Christensen⁷

Affiliations

¹ The Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark.
² Research Unit of Musculoskeletal Function and Physiotherapy, Institute of Sports Science and Clinical Biomechanics, Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark; Department of Physiotherapy and Occupational Therapy, University Hospital of Copenhagen, Herlev and Gentofte, Gentofte, Denmark.
³ Centre for Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway.
⁴ The Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; Department of Physical and Occupational Therapy, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark.
⁵ Center for Evidence-Based Medicine, University of Southern Denmark/Odense University Hospital, Odense, Denmark.
⁶ Department of Psychology, Health and Technology, University of Twente, Enschede, The Netherlands.
⁷ The Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; Research Unit of Rheumatology, Department of Clinical Research, University of Southern Denmark, Odense University Hospital, Odense, Denmark. Electronic address: Robin.Christensen@regionh.dk.

PMID: 32217079
DOI: 10.1016/j.jclinepi.2019.10.016

Abstract

Objective: To compare estimated treatment effects of physical therapy (PT) between patient-reported outcome measures (PROMs) and outcomes measured in other ways.

Study design and setting: We selected randomized trials of PT with both a PROM and a non-PROM included in Cochrane systematic reviews (CSRs). Two reviewers independently extracted data and risk-of-bias assessments. Our primary outcome was the ratio of odds ratios (RORs), used to quantify how effect varies between PROMs and non-PROMs; an ROR > 1 indicates larger effect when assessed by using PROMs. We used REML-methods to estimate associations of trial characteristics with effects and between-trial heterogeneity.

Results: From 90 relevant CSRs, 205 PT trials were included. The summary ROR across all the comparisons was not statistically significant (ROR, 0.88 [95% CI: 0.70-1.12]; P = 0.30); however, the heterogeneity was substantial (I² = 88.1%). When stratifying non-PROMs further into clearly objective non-PROMs (e.g., biomarkers) and other non-PROMs (e.g., aerobic capacity), the PROMs appeared more favorable than did clearly objective non-PROMs (ROR, 1.92 [95% CI: 0.99-3.72]; P = 0.05).

Conclusion: Estimated treatment effects based on PROMs are generally comparable with treatment effects measured in other ways. However, in our study, PROMs indicate a more favorable treatment effect compared with treatment effects based on clearly objective outcomes.

Keywords: Effect size; Meta-Research; Meta-epidemiology; Patient-involvement; Patient-reported outcome measures; Physical therapy.

Publication types

Research Support, Non-U.S. Gov't
Review

MeSH terms

Humans
Meta-Analysis as Topic*
Outcome Assessment, Health Care / statistics & numerical data
Patient Reported Outcome Measures*
Physical Therapy Modalities / statistics & numerical data*
Randomized Controlled Trials as Topic / methods*
Randomized Controlled Trials as Topic / statistics & numerical data*