Purpose: Standardized patient instructors (SPIs) have been used in medical and physician assistant (PA) teaching and have been shown to be a valuable resource for assessing interviewing and clinical skills. This study evaluated the agreement between SPI ratings and student self-ratings in the assessment of counseling skills.
Methods: PA students in three graduating classes (2009-2011) participated in this study. SPIs received specialized training to simulate a patient with diabetes. SPIs provided feedback during simulated encounters using the Diabetes Risk Factor Interview Scale (DRFIS). The DRFIS provides a criteria-based scale for student assessment. Students completed the DRFIS as a self-assessment prior to receiving SPI feedback. Agreement between SPI evaluation and student self-evaluation using the DRFIS were evaluated.
Results: The total DRFIS score was generally comparable between all three graduation classes. SPI ratings of students by individual DRFIS items for all three classes indicate the highest scores occurred for Rapport and Empathy items while the lowest scores were evident for items including Reinforce Effort and Reframing Failure. The percent agreement between SPI and student self-rating was within one point for greater than 70% for all 12 DRFIS items for the cohort of three graduation classes.
Conclusion: PA students rated themselves on average lower on the DRFIS than SPIs but within an acceptable overall percentage range. Using an evaluation scale that is focused around patient counseling skills is a viable teaching tool for students and SPIs. Acceptable agreement between students and SPI was found. The DRFIS provides an effective criteria-based scale for student assessment.