Assessing first year radiology resident competence pre-call: development and implementation of a computer-based exam before and after the 12 month training requirement

Abstract

Rationale and objectives: Whether first-year radiology residents are ready to start call after 6 or 12 months has been a subject of much debate. The purpose of this study was to establish an assessment that would evaluate the call readiness of first-year radiology residents and identify any individual areas of weakness using a comprehensive computerized format. Secondarily, we evaluated for any significant differences in performance before and after the change in precall training requirement from 6 to 12 months.

Materials and methods: A list of >140 potential emergency radiology cases was given to first-year radiology residents at the beginning of the academic year. Over 4 years, three separate versions of a computerized examination were constructed using hyperlinked PowerPoint presentations and given to both first-year and second-year residents. No resident took the same version of the exam twice. Exam score and number of cases failed were assessed. Individual areas of weakness were identified and remediated with the residents. Statistical analysis was used to evaluate exam score and the number of cases failed, considering resident year and the three versions of the exam.

Results: Over 4 years, 17 of 19 (89%) first-year radiology residents passed the exam on first attempt. The two who failed were remediated and passed a different version of the exam 6 weeks later. Using the oral board scoring system, first-year radiology residents scored an average of 70.7 with 13 cases failed, compared to 71.1 with eight cases failed for second-year residents who scored statistically significantly higher. No significant difference was found in first-year radiology resident scoring before and after the 12-month training requirement prior to call.

Conclusions: An emergency radiology examination was established to aid in the assessment of first-year radiology residents' competency prior to starting call, which has become a permanent part of the first-year curriculum. Over 4 years, all first-year residents were ultimately judged ready to start call. Of the variables assessed, only resident year showed a significant difference in scoring parameters. In particular, length of training prior to taking call showed no significant difference. Areas of weakness were identified for further study.