Validating the evaluation capacity scale among practitioners in non-governmental organizations

Steven Sek-Yum Ngai; Chau-Kiu Cheung; Yunjun Li; Lifen Zhao; Lin Wang; Shan Jiang; Hon-Yin Tang; Elly Nga-Hin Yu

doi:10.3389/fpsyg.2022.1082313

Validating the evaluation capacity scale among practitioners in non-governmental organizations

Front Psychol. 2022 Dec 23:13:1082313. doi: 10.3389/fpsyg.2022.1082313. eCollection 2022.

Authors

Steven Sek-Yum Ngai¹, Chau-Kiu Cheung², Yunjun Li¹, Lifen Zhao³, Lin Wang⁴, Shan Jiang⁵, Hon-Yin Tang¹, Elly Nga-Hin Yu¹

Affiliations

¹ Department of Social Work, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China.
² Department of Social and Behavioral Sciences, City University of Hong Kong, Kowloon, Hong Kong SAR, China.
³ Department of Social Security, Nanjing Normal University, Nanjing, China.
⁴ Department of Social Work, Fudan University, Shanghai, China.
⁵ Department of Sociology, Zhejiang University, Hangzhou, China.

Abstract

The growing emphasis on demonstrating the effectiveness of social services through evaluation has heightened demand for nongovernmental organization (NGO) practitioners to enhance evaluation capacity. However, a lack of validated instruments in the NGO context has hampered efforts to assess NGO practitioners' current evaluation capacity and understand how capacity-building activities could be tailored to meet NGO practitioners' actual needs and enhance their evaluation capacity. Hence, this study aims to develop the Evaluation Capacity Scale (ECS), a self-reporting instrument of NGO practitioners' capacity to conduct an effective evaluation of their service programs. Validation data was derived from 439 NGO practitioners who attended the Jockey Club MEL Institute Project in Hong Kong, China. Exploratory factor analysis of the ECS revealed three factors-evaluation mindset, evaluation implementation, and evaluation communication-and confirmatory factor analysis further validated this three-factor structure. Moreover, MANCOVA analysis demonstrated the ECS's predictive validity. Overall, the ECS demonstrated satisfactory convergent validity, high internal consistency reliability, and predictive validity, and its factor structure was supported in subgroups based on gender, age, and level of education. Theoretical and practical implications of the findings are discussed.

Keywords: Hong Kong; assessment of evaluation capacity; internal evaluation; non-governmental organization; scale validation.