Reporting bias in the literature on the associations of health-related behaviors and statins with cardiovascular disease and all-cause mortality

PLoS Biol. 2018 Jun 18;16(6):e2005761. doi: 10.1371/journal.pbio.2005761. eCollection 2018 Jun.

Abstract

Reporting bias in the literature occurs when there is selective revealing or suppression of results, influenced by the direction of findings. We assessed the risk of reporting bias in the epidemiological literature on health-related behavior (tobacco, alcohol, diet, physical activity, and sedentary behavior) and cardiovascular disease mortality and all-cause mortality and provided a comparative assessment of reporting bias between health-related behavior and statin (in primary prevention) meta-analyses. We searched Medline, Embase, Cochrane Methodology Register Database, and Web of Science for systematic reviews synthesizing the associations of health-related behavior and statins with cardiovascular disease mortality and all-cause mortality published between 2010 and 2016. Risk of bias in systematic reviews was assessed using the ROBIS tool. Reporting bias in the literature was evaluated via small-study effect and excess significance tests. We included 49 systematic reviews in our study. The majority of these reviews exhibited a high overall risk of bias, with a higher extent in health-related behavior reviews, relative to statins. We reperformed 111 meta-analyses conducted across these reviews, of which 65% had statistically significant results (P < 0.05). Around 22% of health-related behavior meta-analyses showed small-study effect, as compared to none of statin meta-analyses. Physical activity and the smoking research areas had more than 40% of meta-analyses with small-study effect. We found evidence of excess significance in 26% of health-related behavior meta-analyses, as compared to none of statin meta-analyses. Half of the meta-analyses from physical activity, 26% from diet, 18% from sedentary behavior, 14% for smoking, and 12% from alcohol showed evidence of excess significance bias. These biases may be distorting the body of evidence available by providing inaccurate estimates of preventive effects on cardiovascular and all-cause mortality.

Publication types

  • Meta-Analysis
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Cardiovascular Diseases / mortality*
  • Cardiovascular Diseases / prevention & control*
  • Health Behavior*
  • Health Risk Behaviors
  • Humans
  • Hydroxymethylglutaryl-CoA Reductase Inhibitors / adverse effects
  • Hydroxymethylglutaryl-CoA Reductase Inhibitors / therapeutic use*
  • Primary Prevention* / statistics & numerical data
  • Publication Bias* / statistics & numerical data
  • Risk Factors
  • Systematic Reviews as Topic

Substances

  • Hydroxymethylglutaryl-CoA Reductase Inhibitors

Grants and funding

Sao Paulo Research Foundation (FAPESP) http://www.fapesp.br/en/ (grant number #2014/25614-4 and #2016/21390-0). Leandro Fórnias Machado de Rezende receives doctoral scholarship from Sao Paulo Research Foundation (FAPESP). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. University of Sydney (grant number U2334). Juan Pablo Rey-López is supported by a Postdoctoral fellowship from the University of Sydney (Australia). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. National Health and Medical Research Council (Australia) (grant number). Emmanuel Stamatakis is supported by the National Health and Medical Research Council (Australia) through a Senior Research Fellowship and partly supported by a University of Sydney SOAR Fellowship. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.