Performance of an objective structured clinical examination in a national certification process of trainees in rheumatology

Abstract

Objective: To assess reliability and validity of the objectively-structured clinical examination (OSCE) applied in postgraduate certification processes by the Mexican Board of Rheumatology.

Method: Thirty-two (2013) and 38 (2014) Rheumatology trainees (RTs) underwent an OSCE consisting of 12 and 15 stations respectively, scored according to a validated check-list, as well as 300-multiple-choice 300 question examination (MCQ). Previously, 3 certified rheumatologists underwent a pilot-OSCE. A composite OSCE score was obtained for each participant and its performance examined.

Results: In 2013, OSCE mean score was 7.1±0.6 with none RT receiving a failing score while the MCQ score was 6.5±0.6 and 7 (21.9%) RTs receiving a failing (< 6) score. In 2014, the OSCE score was 6.7±0.6, with 3 (7.9%) RTs receiving a failing score (2 of them also failed MCQ) while the MCQ score was 6.4±0.5 and 7 (18.5%) RTs were disqualified (2 of them also failed OSCE). A significant correlation between the MCQ and the OSCE scores was observed in the 2013 (r=0.44; P=0.006). Certified rheumatologists performed better than RTs at both OSCE. Overall, 86% of RTs obtaining an OSCE passing score also obtained a MCQ passing score, while this was only 67% (P=.02) among those who obtained an OSCE failing score. Nine stations were applied at both consecutive years. Their performance was similar in both certification processes, with correlation coefficients ranging from 0.81 to 0.95 (P≤0.01).

Conclusion: The OSCE is a valid and reliable tool to assess the Rheumatology clinical skills in RTs.

Keywords: Certificación, Reumatología; Certification, Rheumatology; Desempeño clínico; Examen clínico objetivo estructurado; Objectively-structured clinical examination; Trainee.