[Is the formal quality of multiple choice questions used for continuing medical education influenced by guidelines for authors provided by the publishing houses?]

Abstract

Introduction: The widespread use of flawed multiple choice questions (MCQs) in continuing medical education (CME) has been demonstrated in different studies. In recent years, quality control measures have been established by some German publishing houses to ensure the quality of MCQs.

Objectives: The purpose of the present study is to evaluate whether the quality of author's guidelines and the quality of MCQs correlate and whether the quality of MCQs in CME has improved following the implementation of quality control measures.

Materials and methods: A set of 720 items from 6 journals was selected for analysis. A list of 10 item-writing flaws provided by Kühne-Eversmann et al. was modified. The existing guidelines for authors of the six selected journals were categorized by their comprehensiveness.

Results: There were substantial differences in item quality with the percentage of flawed items ranging between 34 % and 92 % of all items published by a journal. One journal showed considerable improvement of item quality following the implementation of guidelines for authors (61 % flawed items in 2006 vs. 33% in 2012). The comprehensiveness of the author's guidelines correlated negatively with the number of item-writing flaws.

Conclusions: The correlation between the existence of elaborate guidelines for authors and the absence of item-writing flaws in MCQs suggests that the publication of guidelines for authors could result in a higher quality of MCQs, even though the review process might have a major influence.

Keywords: Autorenhinweise; Fehler; Multiple-Choice-Fragen; Qualitätsmaßnahmen; author guidelines; continuing medical education (CME); item-writing flaws; multiple choice questions (MCQs); quality control measures; Ärztliche Fortbildung.