[Validity and reproducibility of two direct observation assessment forms for evaluation of internal medicine residents' clinical skills]

Abstract

Introduction: The revision of the French medical studies' third cycle ought to be competency-based. In internal medicine, theoretical and practical knowledge will be assessed online with e-learning and e-portfolio. In parallel, a reflection about clinical skills assessment forms is currently ongoing. In this context, our aim was to assess the reproducibility and validity of two assessment forms based on direct clinical observation.

Method: A prospective and multicentric study has been conducted from November 2015 to October 2016 aiming at evaluating the French translations of the MINI-Clinical Examination Exercice (MINI-CEX) and the Standardized Patient Satisfaction Questionnaire (SPSQ). Included residents have been assessed 2 times over a period of 6 months by the same binoma of judges.

Results: Nineteen residents have been included. The inter-judge reproducibility was satisfactory for the MINI-CEX: intraclass coefficients (ICC) between 0.4 and 0.8 and moderate for the SPSQ: ICC between 0.2 and 0.7 with a good internal coherence for both questionnaires (Cronbach between 0.92 and 0.94). Significant differences between the distributions of the scores given by the judges and a significant inter-center variability have been found.

Conclusion: If the absolute value of the scores should not be taken into account in the evaluation process given its high variability, it could be of interest for the follow-up of the progression in the competencies. These forms could support the residents' debriefing based on the general trends given by the scores.

Keywords: Assessment; Compétence; Internal medicine; Médecine interne; Psychometric study; Skills; Étude psychométrique; Évaluation.