Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children's health: a methodological survey

Sophie K F Michel; Aishwarya Atmakuri; Ondine S von Ehrenstein

doi:10.1186/s12940-024-01069-z

Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children's health: a methodological survey

Environ Health. 2024 Mar 28;23(1):32. doi: 10.1186/s12940-024-01069-z.

Authors

Sophie K F Michel¹, Aishwarya Atmakuri², Ondine S von Ehrenstein^{3

4}

Affiliations

¹ Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA), 650 Charles E Young Dr S, Los Angeles, CA, 90095, USA. sophiemichel@g.ucla.edu.
² Department of Molecular, Cell, and Developmental Biology, University of California, Los Angeles (UCLA), Los Angeles, CA, USA.
³ Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA), 650 Charles E Young Dr S, Los Angeles, CA, 90095, USA.
⁴ Department of Community Health Sciences, Fielding School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, CA, USA.

Abstract

Background: Translating findings from systematic reviews assessing associations between environmental exposures and reproductive and children's health into policy recommendations requires valid and transparent evidence grading.

Methods: We aimed to evaluate systems for grading bodies of evidence used in systematic reviews of environmental exposures and reproductive/ children's health outcomes, by conducting a methodological survey of air pollution research, comprising a comprehensive search for and assessment of all relevant systematic reviews. To evaluate the frameworks used for rating the internal validity of primary studies and for grading bodies of evidence (multiple studies), we considered whether and how specific criteria or domains were operationalized to address reproductive/children's environmental health, e.g., whether the timing of exposure assessment was evaluated with regard to vulnerable developmental stages.

Results: Eighteen out of 177 (9.8%) systematic reviews used formal systems for rating the body of evidence; 15 distinct internal validity assessment tools for primary studies, and nine different grading systems for bodies of evidence were used, with multiple modifications applied to the cited approaches. The Newcastle Ottawa Scale (NOS) and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework, neither developed specifically for this field, were the most commonly used approaches for rating individual studies and bodies of evidence, respectively. Overall, the identified approaches were highly heterogeneous in both their comprehensiveness and their applicability to reproductive/children's environmental health research.

Conclusion: Establishing the wider use of more appropriate evidence grading methods is instrumental both for strengthening systematic review methodologies, and for the effective development and implementation of environmental public health policies, particularly for protecting pregnant persons and children.

Keywords: Air pollution; Body of evidence; Evidence grading; Methodological survey; Reproductive and children’s health; Systematic reviews.

MeSH terms

Air Pollution*
Child
Child Health*
Environmental Exposure / analysis
Female
Humans
Pregnancy
Reproduction
Systematic Reviews as Topic