Purpose: Evaluation of non-cognitive skills never has been used in Brazil. This study aims to evaluate Multiple Mini Interviews (MMI) in the admission process of a School of Medicine in São Paulo, Brazil.
Methods: The population of the study comprised 240 applicants summoned for the interviews, and 96 raters. MMI contributed to 25% of the applicants' final grade. Eight scenarios were created with the aim of evaluating different non-cognitive skills, each one had two raters. At the end of the interviews, the applicants and raters described their impressions about MMI. The reliability of the MMI was analyzed using the Theory of Generalization and Many-Facet Rasch Model (MFRM).
Results: The G-study showed that the general reliability of the process was satisfactory (coefficient G = 0.743). The MMI grades were not affected by the raters' profile, time of interview (p = 0.715), and randomization group (p = 0.353). The Rasch analysis showed that there was no misfitting effects or inconsistent stations or raters. A significant majority of the applicants (98%) and all the raters believed MMIs were important in selecting students with a more adequate profile to study medicine.
Conclusions: The general reliability of the selection process was excellent, and it was fully accepted by the applicants and raters.