Analysis of the French national evaluation of radiology residents

Abstract

Purpose: In France, a national evaluation is given annually to radiology residents. The aim of this study was to perform both a docimological analysis of the quality of the questionnaire and a statistical analysis of the results.

Materials and methods: This retrospective study, which included French radiology residents from Year 1 to Year 5 of residency, was performed from 2015 to 2017 across 25 medical universities in France. Both qualitative and quantitative docimological analyses were performed as assessed by the Cronbach alpha coefficient, the difficulty of question (PDI), and the coefficient of discrimination (Rir). Results to the questionnaire were compared between years of residency.

Results: The results of the analysis confirmed the quality of the questionnaire (Cronbach alpha coefficient=0.71, mean [PDI=0.40]) though the majority of questions could be answered by memory rather than cognitive ability. The mean Rir was 0.02, indicating that students could not be certified using only the questionnaire. The results measuring resident level of knowledge were moderate, with mean results ranging from 9.2/20 at the first year to 11.3/20 at the fifth year of residency (P<0.001). There were no significant differences in results obtained between the third, fourth, and fifth year of residency but results were significantly different among university hospitals.

Conclusion: Even if close interactions exist between learning and pedagogic environment, our results suggest that it may be useful to further develop an evaluation process in relation with pedagogic instructions in order to provide more optimal training.

Keywords: Docimology; Learning; Pedagogy; Radiology.