Implementation and outcome evaluations of a multi-site improvement program in cancer genetics

Abstract

Program evaluation can identify the successes and challenges of implementing clinical programs, which can inform future dissemination efforts. A cancer genetics improvement program, disseminated from the Lead Team's institution to five health systems (Participating Sites), was genetic counselor led, using virtual implementation facilitation to support Participating Sites' performance of quality improvement (QI) activities over several years. Program implementation and outcome evaluations were performed and included evaluation of program delivery and initial effects of the program on Participating Sites. A logic model guided evaluation of program implementation (inputs, activities, outputs, delivery/fidelity, and coverage/reach) and initial outcomes (short-term and intermediate outcomes). Data were collected from program documents and an Evaluation Survey of Participating Site team members (21 respondents), compared against the Lead Team's expectations of participation, and analyzed using descriptive statistics. All program inputs, outputs, and activities were available and delivered as expected across the five Participating Sites. The most frequently used activities and inputs were facilitation-associated meetings and meeting resources, which were rated as useful/helpful by the majority of respondents. Nearly all respondents noted improvement in short-term outcomes following participation: 82.4% reported increased awareness of clinical processes, 94.1% increased knowledge of QI methods, 100% reported increased perceived importance of QI, 94.1% increased perceived feasibility of QI, and 76.5% reported increased problem-solving skills and self-efficacy to use QI at their site. Intermediate outcomes (identifying barriers, developing interventions, improved teamwork, and capacity) were achieved following program participation as indicated by the results of the program document review and Evaluation Survey responses. Implementation challenges at Participating Sites included staffing constraints, difficulties obtaining buy-in and participation, and developing interventions over time. The multi-site improvement program was delivered and implemented with high levels of fidelity and resulted in improved short and intermediate outcomes. Future research will evaluate long-term, patient-level outcomes associated with site-specific QI interventions.

Keywords: genetic counseling; genetic testing; genetics services; program evaluation.