Exploring the impact of evaluation on learning and health innovation sustainability: protocol for a realist synthesis

Marissa Bird; Élizabeth Côté-Boileau; Walter P Wodchis; Lianne Jeffs; Maura MacPhee; James Shaw; Tujuanna Austin; Frances Bruno; Megan Bhalla; Carolyn Steele Gray

doi:10.1186/s13643-023-02348-5

Exploring the impact of evaluation on learning and health innovation sustainability: protocol for a realist synthesis

Syst Rev. 2023 Oct 6;12(1):188. doi: 10.1186/s13643-023-02348-5.

Authors

Marissa Bird^{1

2}, Élizabeth Côté-Boileau³, Walter P Wodchis^{4

5}, Lianne Jeffs⁶, Maura MacPhee⁷, James Shaw⁸, Tujuanna Austin⁵, Frances Bruno^{5

6}, Megan Bhalla⁹, Carolyn Steele Gray^{5

6}

Affiliations

¹ Institute for Better Health, 100 Queensway West-Clinical, Administrative Building, 6Th Floor, Mississauga, ON, L5B 1B8, Canada. marissa.bird@thp.ca.
² Institute of Health Policy Management and Evaluation (IHPME), Dalla Lana School of Public Health, University of Toronto, Suite 425-155 College Street, Toronto, ON, M5T 3M6, Canada. marissa.bird@thp.ca.
³ Department of Health Management, Evaluation and Policy, School of Public Health, University of Montreal, 7101 Av du Parc, Montréal, QC, H3N 1X9, Canada.
⁴ Institute for Better Health, 100 Queensway West-Clinical, Administrative Building, 6Th Floor, Mississauga, ON, L5B 1B8, Canada.
⁵ Institute of Health Policy Management and Evaluation (IHPME), Dalla Lana School of Public Health, University of Toronto, Suite 425-155 College Street, Toronto, ON, M5T 3M6, Canada.
⁶ Science of Care Institute, Sinai Health, 1 Bridgepoint Drive, Toronto, ON, M4M 2B5, Canada.
⁷ Nursing-Applied Sciences, University of British Columbia, 239-2211 Wesbrook Mall, Vancouver, BC, V6T2B5, Canada.
⁸ Department of Physical Therapy, Temerty Faculty of Medicine, University of Toronto, 500 University Ave, Toronto, ON, M5G 1V7, Canada.
⁹ OPTI-Hex Lab, Leslie Dan Faculty of Pharmacy, University of Toronto, 144 College St, Toronto, ON, M5S 3M2, Canada.

Abstract

Background: Within the Learning Health System (LHS) model, learning routines, including evaluation, allow for continuous incremental change to take place. Within these learning routines, evaluation assists in problem identification, data collection, and data transformation into contextualized information, which is then re-applied to the LHS environment. Evaluation that catalyzes learning and improvement may also contribute to health innovation sustainability. However, there is little consensus as to why certain evaluations seem to support learning and sustainability, while others impede it. This realist synthesis seeks to understand the contextual factors and underlying mechanisms or drivers that best support health systems learning and sustainable innovation.

Methods: This synthesis will be guided by Pawson and colleagues' 2005 and Emmel and colleagues' 2018 guidelines for conducting realist syntheses. The review process will encompass five steps: (1) scoping the review, (2) building theories, (3) identifying the evidence, (4) evidence selection and appraisal, and (5) data extraction and synthesis. An Expert Committee comprised of leaders in evaluation, innovation, sustainability, and realist methodology will guide this synthesis. Review findings will be reported using the RAMESES guidelines.

Discussion: The use of a realist review will allow for exploration and theorizing about the contextual factors and underlying mechanisms that make evaluations 'work' (or 'not work') to support learning and sustainability. Depending on results, we will attempt to synthesize findings into a series of recommendations for evaluations with the intention to support health systems learning and sustainability. Finalized results will be presented at national and international conferences, as well as disseminated via a peer-reviewed publication.

Systematic review registration: This realist synthesis protocol has been registered with PROSPERO ( https://www.crd.york.ac.uk/prospero/ ID 382690).

Keywords: Evaluation; Innovation; Learning; Learning health systems; Realist synthesis; Sustainability.

MeSH terms

Humans
Learning*
Research Design*
Systematic Reviews as Topic