To evaluate the efficacy of Multiple Choice Questions (MCQ) as an evaluation tool in medical education, were analyzed the responses to 80 MCQ delivered by 300 physicians working in emergency departments and following a course on Basic and Advanced Life Support. Pre- and post-test were administered using a dedicated computer application running on PC. Students' scores were high and demonstrated both course and test validity. More than 95% of the students used the computer application without interface difficulties. However, some limits of MCQ were individuated: first, misinterpretation of some questions, especially when two or more similar answers were presented; from the other side, the student's personal experiences during the course practice could influence student's responses bringing him to different conclusions on respect to the instructor. These difficulties should be kept in mind in the development of a MCQ session, and students should have the possibility to give to the instructor a feedback of problems encountered during the course. This is especially important when MCQ are the sole evaluation technique.