Evaluating a learning health system initiative: Lessons learned during COVID-19 in Saskatchewan, Canada

Gary Groot; Stephanie Witham; Andreea Badea; Susan Baer; Michelle Dalidowicz; Bruce Reeder; John Froh; Tracey Carr

doi:10.1002/lrh2.10350

Evaluating a learning health system initiative: Lessons learned during COVID-19 in Saskatchewan, Canada

Learn Health Syst. 2022 Oct 9;7(3):e10350. doi: 10.1002/lrh2.10350. Online ahead of print.

Authors

Gary Groot^{1

2}, Stephanie Witham¹, Andreea Badea¹, Susan Baer³, Michelle Dalidowicz³, Bruce Reeder¹, John Froh², Tracey Carr¹

Affiliations

¹ Department of Community Health and Epidemiology University of Saskatchewan Saskatoon Saskatchewan Canada.
² Saskatchewan Health Authority Royal University Hospital Saskatoon Saskatchewan Canada.
³ Saskatchewan Health Authority Health Sciences Library Regina Saskatchewan Canada.

Abstract

Introduction: Evaluating a learning health system (LHS) encourages continuous system improvement and collaboration within the healthcare system. Although LHS is a widely accepted concept, there is little knowledge about evaluating an LHS. To explore the outputs and outcomes of an LHS model, we evaluated the COVID-19 Evidence Support Team (CEST) in Saskatchewan, Canada, an initiative to rapidly review scientific evidence about COVID-19 for decision-making. By evaluating this program during its formation, we explored how and to what extent the CEST initiative was used by stakeholders. An additional study aim was to understand how CEST could be applied as a functional LHS and the value of similar knowledge-to-action cycles.

Methods: Using a formative evaluation design, we conducted qualitative interviews with key informants (KIs) who were involved with COVID-19 response strategies in Saskatchewan. Transcripts were analyzed using reflexive thematic analysis to identify key themes. A program logic model was created to represent the inputs, activities, outputs, and outcomes of the CEST initiative.

Results: Interview data from 11 KIs were collated under three overarching categories: (1) outputs, (2) short-term outcomes, and (3) long-term outcomes from the CEST initiative. Overall, participants found the CEST initiative improved speed and access to reliable information, supported and influenced decision-making and public health strategies, leveraged partnerships, increased confidence and reassurance, and challenged misinformation. Themes relating to the long-term outcomes of the initiative included improving coordination, awareness, and using good judgment and planning to integrate CEST sustainably into the health system.

Conclusion: This formative evaluation demonstrated that CEST was a valued program and a promising LHS model for Saskatchewan. The future direction involves addressing program recommendations to implement this model as a functional LHS in Saskatchewan.

Keywords: COVID‐19; Saskatchewan; decision‐making; evaluation; health services research; learning health system.