Background: To improve the admissions process for the Université de Montréal (UdeM) ophthalmology residency program, the interview structure was modified to encompass the seven CanMEDS roles introduced by the Royal College of Physicians and Surgeons of Canada (RCPSC). These roles include an applicant's abilities as a communicator, collaborator, manager, health advocate, professional, scholar, and medical expert.
Methods: In this retrospective pilot study, the records of all applicants were reviewed by 8 members of the admissions committee, with a high intraclass correlation coefficient of 0.814. Four 2-person interview teams were then formed. The first 3 groups asked the applicants specific questions based on 2-3 of the CanMEDS roles, marking their impressions of each candidate on a visual analogue scale. The last group answered candidates' questions about the program but assigned no mark.
Results: The intraclass correlations for the teams were 0.900, 0.739, and 0.585, demonstrating acceptable interrater reliability for 2 of the teams. Pearson correlation coefficients between groups of interviewers were considered adequate at 0.562, 0.432, and 0.417 (p < 0.05). For each interviewer, the Pearson correlation coefficient between record marking and interview scoring was either not statistically significant or very low.
Interpretation: By basing the 2006 interview process on the CanMEDS roles defined by the RCPSC, information was obtained about the candidates that could not have been retrieved by a review of the medical students' records alone. Reliability analysis confirmed that this new method of conducting interviews provided sound and reliable judging and rating consistency between all members of the admissions committee.