Validity and reliability of a novel written examination to assess knowledge and clinical decision making skills of medical students on the surgery clerkship

Abstract

Background: The Surgery Clerkship Clinical Skills Examination (CSE) is a novel written examination developed to assess the surgical knowledge, clinical decision making, communication skills, and professionalism of medical students on the surgery clerkship. This study was undertaken to determine its validity.

Methods: Data were prospectively collected from July 2011 through February 2013. Multivariate linear and logistic regression analyses were used to assess score trend; convergent validity with National Board of Medical Examiners surgery and medicine subject scores, United States Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge scores, and evaluation of clinical reasoning and fund of knowledge; and the effect of clerkship order. Exam reliability was assessed using a modified Cronbach's α statistic.

Results: During the study period, 262 students completed the CSE, with a normal distribution of performance. United States Medical Licensing Examination Step 2 Clinical Knowledge score and end-of-clerkship evaluations of fund of knowledge and clinical reasoning predicted CSE score. Performance on the CSE was independent of clerkship order or prior clerkships. The modified Cronbach's α value for the exam was .67.

Conclusions: The CSE is an objective, valid, reliable instrument for assessing students on the surgery clerkship, independent of clerkship order.

Keywords: Assessment; Clinical reasoning; Clinical skills; Medical student; Objective examination; Surgical education.