Purpose: This research involved the development, and in particular the evaluation of the reliability, of scales to measure medical faculty attitudes toward clinical evaluation (ACE) of medical students. The intent was to create measures that yield reliable data and have practical utility in medical education research and faculty development.
Method: A systematic, eight-step scale development protocol was used to create the instrument. In early 1993 factor analysis was used on data from 217 clinical faculty at four medical schools to refine the measures. Internal consistency and test-retest reliability analyses were performed. Analyses were also done to determine whether the attitude scores were influenced by such faculty demographic attributes as employing medical school, gender, age, tenure track status, academic rank, or academic department.
Results: An initial pool of 52 items was reduced to 30 items based on iterative reliability studies. Factor analysis on the 30 items yielded two scales: (1) Quality of Evaluation Procedures, 12 items, alpha = .81; and (2) Content of Departmental Evaluations, eight items, alpha = .85. Test-retest reliabilities (12 weeks) for the scales were .67 and .74, respectively. Faculty demographics did not influence attitudes about the quality of evaluation procedures. However, family physicians showed a slightly more positive attitude toward the content of departmental evaluations than did physicians in five other medical specialties.
Conclusion: The goal of developing reliable measures of faculty attitudes toward clinical evaluation of medical students has been achieved. With baseline reliabilities established, future research should assess the validity and utility of the scales, especially in the context of clinical practice examinations.