An exploration of synthesis methods in public health evaluations of interventions concludes that the use of modern statistical methods would be beneficial

Abstract

Objectives: To review the methods currently used to synthesize evidence in public health evaluations and demonstrate the availability of more sophisticated approaches.

Study design and setting: A systematic review of National Institute for Health and Care Excellence (NICE) public health appraisals published between 2006 and 2012 was performed to assess the methods used for the synthesis of effectiveness evidence. The ability of new developments in evidence synthesis methodology to address the challenges and opportunities present in a public health context is demonstrated.

Results: Nine (23%) of the 39 NICE appraisals included in the review performed pairwise meta-analyses as part of the effectiveness review with one of these also including a network meta-analysis. Of the remainder, 29 (74.4%) presented narrative summaries of the evidence only, and 1 (2.6%) appraisal did not present any review of effectiveness and/or cost-effectiveness evidence. Heterogeneity of outcomes, methods, and interventions were the main reasons given for not pooling the data. Exploration of quantitative synthesis methods shows that pairwise meta-analyses can be extended to incorporate individual participant data (when it is available), extend the number of interventions being compared using a network meta-analysis, and adjust for both subject- and summary-level covariates. All these can contribute to ensuring the analysis answers directly the policy-relevant questions.

Conclusion: More sophisticated methods in evidence synthesis should be considered to make evaluations in public health more useful for decision makers.

Keywords: Decision making; Meta-analysis; Network meta-analysis; Public health evaluation; Systematic review.