Inadvertent P-hacking among trials and systematic reviews of the effect of progestogens in pregnancy? A systematic review and meta-analysis

Abstract

Background: Progestogens have been evaluated in numerous trials and meta-analyses, many of which concluded they were effective. However, two large trials PROMISE and OPPTIMUM have recently concluded that progesterone was ineffective. This raises the possibility that earlier studies and reviews had been biased by either selective publication or selective choice of outcomes, so called "P-hacking".

Objectives: To compare the findings all progestogen trials and systematic reviews with those of trials with pre-registered primary outcomes which avoided selective outcome reporting.

Search strategy: Search of PubMed, the Cochrane Library and trial registries. Registration PROSPERO CRD42016035303.

Selection criteria: Systematic reviews of randomised trials comparing progestogen with placebo in pregnancy and the individual trials included in those reviews. The subset of trials reporting a pre-registered primary outcome were compared with the totality of trials and reviews.

Data collection and analysis: For reviews all outcomes were included. For individual trials all outcomes reported in the systematic reviews were included. For the comparison group we recorded the registered primary outcome from trials that were either registered before they started, or registered during the recruitment phase and also double blind.

Main results: Nineteen of twenty-nine meta-analyses concluded that progestogens were effective. Twenty-two trials reported their pre-registered primary outcomes. There was no effect of progesterone on primary registered dichotomous outcome RR 1.00 (95% CI 0.94-1.07). Only one of the 22 showed a nominally statistically significant benefit.

Author's conclusions: When evaluated in registered double-blind trials with analysis restricted to predefined primary outcomes, progestational agents in pregnancy are ineffective.

Tweetable abstract: Progestogens to prevent pregnancy loss, an example of P-hacking.

Keywords: Miscarriage; P-hacking; outcome switching; pregnancy loss; preterm birth; progestogen.