Psychometric characteristics of the 360° feedback scales in professionalism and interpersonal and communication skills assessment of surgery residents in China

Abstract

Aims: The aim of the study was to test the psychometric characteristics of the 360° evaluation instrument for assessing residents' competency in professionalism, interpersonal and communication skills, and develop a feasible, valid, and reliable multisource feedback (MSF) program for surgery residents to address this gap.

Methods: We carried out an MSF assessment of 149 surgery residents at 19 hospitals all over China. MSF assessment includes 6 surveys with 21, 21, 21, 26, 14, and 15 items and these surveys were developed to assess surgery residents by attending doctor, resident self, peer, nurse, patient, and office staff, respectively, using a 5-point agreement scale with an "unable-to-evaluate" category. Reliability was assessed by Cronbach alpha. Exploratory factor analysis was used to assess validity and determine which items grouped together into scales.

Results: A total of 2384 questionnaires were analyzed in this study. The internal consistency reliability of the instruments was a Cronbach alpha of 0.932, 0.936, 0.914, 0.916, 0.939, and 0.903 for attending doctor, resident self, nurse, patient, resident peer, and office staff surveys, respectively. On the attending, resident self, resident peer, nurse, office staff, and patient surveys, the factor analysis identified 2 factors of professionalism and interpersonal and communication skills accounting for 75.62%, 74.81%, 72.65%, 73.38%, 76.11%, and 63.89% of the variance, respectively. Some items, such as Demonstrates respect for my "Sexual Orientation," "Religion," and "Disability," in different surveys had high unable-to-evaluate rates (more than 10%).

Conclusions: The data suggest that these instruments developed to assess surgery residents are feasible to administer and provide valid and reliable evidence. Some items in survey need to be adjusted keeping in mind the Chinese culture.

Keywords: Interpersonal and Communication Skills; Patient Care; Professionalism; interpersonal and communication skills; multisource feedback; professionalism; resident.