The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments

Nurse Educ Today. 2006 Dec;26(8):662-71. doi: 10.1016/j.nedt.2006.07.006. Epub 2006 Oct 2.

Abstract

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

Publication types

  • Evaluation Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Bias
  • Cognition
  • Comprehension
  • Cues
  • Discriminant Analysis
  • Education, Nursing, Associate* / standards
  • Education, Nursing, Baccalaureate* / standards
  • Education, Professional, Retraining / standards
  • Educational Measurement / methods*
  • Educational Measurement / standards
  • Faculty, Nursing / standards
  • Guideline Adherence / standards
  • Guidelines as Topic
  • Health Services Needs and Demand
  • Hong Kong
  • Humans
  • Linguistics / standards
  • Mental Recall
  • Nursing Education Research
  • Psychometrics
  • Quality Control
  • Students, Nursing / psychology
  • Surveys and Questionnaires / standards*
  • Writing / standards*