Implementation Degree Assessment Sheet for Health Program in Japan by Customizing CFIR: Development and Validation

Reiko Okamoto; Masako Kageyama; Keiko Koide; Miho Tanaka; Yoshiko Yamamoto; Mana Fujioka; Ayami Osuna; Kazuko Saeki; Kazue Hirokane; Fusami Nagano; Shinji Takemura

doi:10.1186/s43058-022-00270-w

Implementation Degree Assessment Sheet for Health Program in Japan by Customizing CFIR: Development and Validation

Implement Sci Commun. 2022 Feb 19;3(1):20. doi: 10.1186/s43058-022-00270-w.

Authors

Affiliations

¹ Division of Health Sciences, Osaka University Graduate School of Medicine, Yamadaoka 1-7, Suita-city, Osaka, 565-0871, Japan. reiko@sahs.med.osaka-u.ac.jp.
² Division of Health Sciences, Osaka University Graduate School of Medicine, Yamadaoka 1-7, Suita-city, Osaka, 565-0871, Japan.
³ Former Division of Health Sciences, Osaka University Graduate School of Medicine, Osaka, Japan.
⁴ Faculty of Nursing, Toyama Prefectural University, Nishi-nagae 2-2-78, Toyama-city, Toyama, 930-0975, Japan.
⁵ School of Nursing, Hyogo University of Health Sciences, Minatojima 1-3-6, Chuo-ku, Kobe-shi, Hyogo, 650-8530, Japan.
⁶ Faculty of Nursing, Fukuyama Heisei University, Kamiiwanari-syoto 117-1, Miyuki-town, Fukuyama-city, Hiroshima, 720-0001, Japan.
⁷ Department of Health Policy and Technology Assessment, National Institute of Public Health, Wako-city-minami 2-3-6, Saitama, 351-0197, Japan.

Abstract

Purpose: More than 70% of public health nurses in Japan belong to government agencies, and there is a need for further evidence-based capacity development for program implementation. The purpose of this research was to develop an Implementation Degree Assessment Sheet (IDAS) by customizing the Consolidated Framework for Implementation Research (CFIR) to health programs in Japan.

Methods: The draft IDAS (five domains, 31 constructs) created by customizing the CFIR was refined by the researchers and modified through pre-testing. The survey covered full-time public health nurses (PHNs) affiliated to all prefectures and the cities with health centers of Japan. The survey was conducted as an anonymous, self-administered questionnaire survey by mail.

Results: One hundred eighty-five of the 334 institutions (55.4%) agreed to participate in our survey. Of the 966 questionnaire forms distributed, 709 forms (73.4%) were collected, of which 702 responses (72.7%) were valid. No item required consideration of deletion based on the results of item analysis, and our confirmatory factor analysis on model fitness between the five IDAS domains and CFIR showed sufficient fit indices after modification. With regard to reliability, Cronbach's coefficient alpha, a measure of internal consistency, stayed above 0.8 overall. Our verification of stability with the split-half (odd/even) method resulted in a Spearman-Brown reliability coefficient of 0.95. The correlation coefficient between the IDAS scores and the research utilization competency score, used as an external criterion, was 0.51 (p<0.001), supporting the coexistence validity of the criterion-related validity of the IDAS. The significant differences were observed between known-groups, supporting the known-group validity of the IDAS.

Conclusion: This study developed the IDAS and confirmed constant reliability and validity. Hereafter, it is necessary to promote the required capacity development based on the actual degree of implementation in order to use the IDAS for the competency development of public health nurses and related professions to deliver health programs.

Keywords: Administrative health program; Assessment tool; Consolidated Framework for Implementation Research (CFIR); Implementation science; Measurement; Public health nurse.

Grants and funding

JP19H03961./Japan Society for the Promotion of Science