Comparing veterinary students' performance with cut-scores determined using a modified individual Angoff method featuring Bloom's taxonomy

Abstract

Background: There are challenges around the practicality of conventional standard setting methods for student assessment. Furthermore, accuracy of absolute methods of standard setting is difficult to achieve.The aim was to determine which group of judges is most accurate at establishing the minimum level required to pass questions in order to ensure an appropriate standard (cut-scores), and how the Bloom's level of each question affected the correlation of cut-scores to student performance.

Methods: The modifications to the classical Angoff method where a group of judges convene and discuss cut-scores was that, in this study, the judges set cut-scores independently and did not receive the answers to the questions that they were assessing. Computer-based multiple choice and multiple response type questions were compiled, and allocated Bloom's levels. Judges answered the questions, determined cut-scores and completed a questionnaire. Simple linear regression was used to determine whether number of years' experience, proportion of time spent in small ruminant practice or specialisation in the field resulted in the most accurate comparison to student performance.

Results: Individuals spending the greatest proportion of time in small ruminant practice demonstrated greater accuracy in determining cut-scores. The Bloom's level assigned to each question was reflected on student performance.

Conclusion: This study supports that the time spent in a particular discipline must be taken into consideration when selecting judges for establishing cut-scores, and that the cognitive level of each exam question be considered to improve accuracy.

Keywords: Bloom's cognitive levels; modified Angoff; predicting student performance; small ruminants; veterinary education; veterinary graduates.